B2G! Fennec! Social API! They’re all happening, and memory consumption is a big deal for all of them. Time for a MemShrink report.
Lots of B2G work is happening, unsurprisingly. All of these changes have been backported to the Aurora channel.
Kyle Huey landed an important patch that merges system compartments together. This avoids wasted space caused by having 100s of small compartments. As Chris Jones said: “It’s a completely different phone with these patches.” Unfortunately, the preference controlling this behaviour is currently turned off on B2G, because it’s causing some Marionette test failures. Hopefully they’ll be dealt with soon. Note that this preference won’t be turned on in desktop builds, and there’s a medium-term plan to avoid this wasted space in a less hacky fashion.
Marco Bonardo disabled Places in B2G, saving about 300 KiB in the main process.
Justin Lebar added code to trigger memory pressure events when lowmemkiller notifications occur.
Justin also tweaked things so that the garbage collector is more likely to run when screenshots are taken.
Jeff Muizelaar fixed a graphics bug that I don’t understand but apparently avoids allocating lots of memory on the B2G unlock screen.
Chris Jones ensured that remote content drop their buffers when they’re hidden.
I tweaked the size of the arena chunks used by XPT, which saved about 120 KiB per process on 64-bit builds, and a bit less on 32-bit builds.
I also shrunk the initial size of the SPS hash table, which saves 48 KiB per process on 64-bit builds and 24 KiB on 32-bit builds.
Lots of work happened on the memory reporting front. This was inspired by “heap-unclassified” typically being much higher on B2G than on desktop. Desktop Firefox uses a single process and its memory consumption is usually measured in the 100s of MiBs or more. In contrast, B2G uses one “main” process and then one process per running app, and the smaller ones are typically around 10 MiB. As a result, the sundry small per-process things that don’t matter much for desktop loom larger on B2G. (For the same reason, changes that reduce per-process memory by smallish amounts have outsize value on B2G.) Most of the following changes have also been backported to the Aurora channel.
Nick Hurley added a “explicit/network/disk-cache” memory reporter. It typically measures 100s of KiBs in desktop Firefox. (It doesn’t get used on B2G.)
I added memory reporters “explicit/xpcom/component-manager” and “explicit/xpcom/category-manager”. Together they measure about 280 KiB per process on 64-bit builds, and a bit less on 32-bit builds.
I added a memory reporter “explicit/script-namespace-manager”. It measures just over 150 KiB per process on 64-bit builds, and a bit less on 32-bit builds.
I added a memory reporter “explicit/xpcom/effective-TLD-service”. It measures about 128 KiB per process on 64-bit builds, and a bit less on 32-bit builds.
I added some detail to the JS object memory reporters. This gives slightly more insight into where this memory is going, but the extra memory measured as a result is usually pretty small.
I fixed the “explicit/atom-tables” memory reporter, which was erroneously always reporting 0 bytes. It now measures anywhere from a few 100 KiBs to several MiBs of memory. This was a frustrating bug to find. DMD exists to check that memory reporters are measuring memory properly, and the reporter was doing its measurements just fine. But DMD cannot check that the measured amounts are added correctly, and that’s what was going wrong here — a line that should have been this:
instead was this:
In other words, the code I had written was doing the difficult part correctly, but botched the trivial part. How annoying.
Felipe Gomes fixed a document leak that occurred when the social sidebar was hidden, and disappeared only when it was unhidden.
Felipe also added code to clear the previous profile when Social is toggled off, which prevents a leak.
The social API still has some significant unresolved memory consumption issues, which are a concern.
Kartikaya Gupta (a.k.a. Kats) turned on tab expiration for Fennec. What this means is that Fennec will now unload the contents of background tabs in certain circumstances — when memory is low, and in some cases when a tab hasn’t been viewed for over an hour — and then reload them when they are brought back into the foreground. Read Kats’ blog post for more details.
(This change was prompted by Project 256meg, which aims to make Fennec usable on phones with only 256 MiB of RAM. (Fennec currently requires 512 MiB, according to the official specs.) Kats has just got to the point where Fennec can actually start on 256 MiB devices. Although Fennec only uses a single process, there is obviously significant overlap between this goal and B2G’s memory reduction goals.)
An obvious follow-up idea is to add tab expiration to desktop Firefox. But the bug tracking that idea has seen more heated discussion. The reason is that tab expiration might cause data loss in some cases. This is less of a problem on Fennec because (as far as I can tell) there is less of an expectation that in-flight data will be saved on mobile devices. For example, having processes killed due to memory constraints on mobile is much more common than on desktop. Still, I expect plenty more arguing before this issue is resolved one way or the other. One option is to allow it on desktop but have it disabled by default.
Michel Gutierrez fixed a zombie compartment in Video DownloadHelper, which is the 2nd most popular add-on at AMO. Version 4.9.11 has the fix.
We hit a notable milestone on the add-on front this fortnight: on November 1st hit bug for tracking known leaks in add-ons — filed 16 months ago — had zero blockers. In other words, we reached the point where no add-ons were known to leak! Unfortunately this didn’t last long, and at the time of writing the tracking bug has three blocking bugs. Still, it’s confirmation that what used to be our biggest memory consumption problem is well under control.
Here are the current bug counts.
- P1: 20 (-3/+5)
- P2: 110 (-10/+7)
- P3: 102 (-3/+8)
- Unprioritized: 4 (-2/+4)
As always, the current bug lists can be found by following the links on the MemShrink wiki page.