Category: B2G

Categories
about:memory B2G Firefox Memory consumption MemShrink

MemShrink progress, week 83–84

Fixed

Justin Lebar made it so that memory reports could be collected on production B2G phones (i.e. not just developer phones with root access).  This was a MemShrink:P1, because getting these reports is crucial.

Gregor Wagner tuned the GC heuristics used by workers.  This is important for B2G, which uses workers extensively.

Andrew McCreight fixed a leak involving audio contexts.

I added a memory reporter for event targets, which includes XHRs.  This can measure multiple MiB of memory when running Gmail.

I added a memory reporter for data held by the JS engine’s regexp JIT compiler.  It usually measures insignificant amounts.

I fixed an inaccuracy in the “resident” memory report tree, which is visible in about:memory when running on Linux, which was caused by a change in recent kernels.

AWSY

The recent results on AWSY have been ugly.  There were two bad regressions in December, as the following graph makes clear.

areweslimyet.com, december 2012

John Schoenick did some work to improve AWSY to make regression hunting easier, and as a result we finally know which changes caused these regressions.

  • A refactoring of images code caused the bigger regression, on December 18.  Seth Fowler is looking into this.
  • Two changes relating to the new DOM bindings caused the smaller regression on December 11/12.  This is largely because many more JS getter/setter functions are present.  It’s not clear yet how to win back this memory, though it should be possible to turn these changes off in the short-term.

These regressions have made it to the Aurora branch, which means there is some urgency now to either fix them or back out/disable them soon.  We don’t want them to reach Beta.

Bug Counts

Here are the current bug counts.

  • P1: 21 (-1/+7)
  • P2: 125 (-2/+11)
  • P3: 104 (-1/+3)
  • Unprioritized: 4 (-17/+3)

The changes are larger than usual because we had a big log of untriaged bugs to go through, due to the six week break since the last MemShrink meeting.

Categories
about:memory B2G DMD Fennec Firefox Memory consumption MemShrink

MemShrink progress, week 79–82

I skipped the last MemShrink report due to Christmas, so we have four weeks’ worth of bug fixes today.

LEAKS

Joe Walker fixed a bad leak found by Jesse Ruderman:  if you closed a browser window with the developer toolbar open it would leak “everything”.  This was a MemShrink P1 bug.

Anton Kovalyov fixed a leak involving scratchpad.  This bug was also found by Jesse Ruderman.

Randell Jesup fixed some WebRTC leaks.

John Schoenick fixed a leak involving plugins.

Josh Aas fixed a leak in some networking code.

DMD

I wrote a more detailed blog post about DMD.  Here is the take-away message.

about:memory is MemShrink’s not-so-secret weapon when it comes to understanding Firefox’s memory consumption… and DMD is how we make about:memory better.

Lots of under-the-hood improvements have been made to DMD since I wrote that.  Users on Mac, Linux and B2G who aren’t afraid of doing their own builds should try it out.  Also, Ehsan Akhgari got it to build on Windows, though it’s not yet clear how well it works on that platform.  If anyone wants to try it out, please let me know how it goes.

Memory Reporters

Ben Turner made the workers memory reporter be able to handle workers that use ctypes.  This was important, especially on B2G, because each process can have one or two or more such workers — this is for Firefox chrome stuff, not web content — and we weren’t measuring them at all, and they can take multiple MiB each.

I fixed the orphan DOM node memory reporter.  The introduction of WebIDL had changed the layout of some paired JS/DOM objects, and such objects weren’t being reported.  (DMD discovered the unreported memory, and Boris Zbarsky helped me interpret what it meant.)  I see this accounting for multiple MiB of orphan nodes when using Gmail.

I added a memory reporter for the event listenener manager’s hash table.  It starts off small, but I’ve seen it go as high as 1.5 MiB after lots of browsing.  (DMD helped me identify this too.)

Kartikaya Gupta added a memory reporter for graphics textures on Android.

I added a memory reporter for any ctypes data that is hanging off JS objects.  I added it because one DMD profile on B2G showed non-trivial amounts of ctypes data, but that seems to have been a fluke and it rarely shows much memory now.  Oh well.

Miscellaneous

Andrew McCreight improved CC shutdown logging, which will make it easier to identify shutdown leaks.

Rail Aliiev and Kartikaya Gupta enabled a new NDK for Fennec builds on releng machines.  This might result in smaller binary sizes, which saves memory.

Bug Counts

Here are the current bug counts.

  • P1: 15 (-2/+0)
  • P2: 116 (-10/+0)
  • P3: 102 (-4/+0)
  • Unprioritized: 18 (-0/+18)

The number of unprioritized bugs is high because we didn’t have a MemShrink meeting this week.  This was because Justin Lebar and Kyle Huey are in Berlin for the B2G work week.  We’ll have our next meeting two weeks from today.

Categories
B2G DMD Firefox Memory consumption MemShrink

MemShrink progress, week 77–78

DMD

The big news this week is the landing of the native version of DMD.  The old version of DMD is a Valgrind-based tool that has been instrumental in reducing the size of about:memory’s “heap-unclassified”.  (For example, with trunk builds on my Linux desktop machine it’s now frequently less than 10%.)

However, the old version of DMD wasn’t easy to run.  For example, it required patching Valgrind’s source code and re-building it, among other things.  As a result, only a handful of people ever ran it.  Furthermore, because it’s a Valgrind tool it is very slow and will never run on Windows.

In contrast, the new version is much easier to use.  It just requires a slight configuration change at build time and a slight change in how the browser is invoked.

It’s also much faster, especially if you use the sampling mode which trades a small amount of precision for a great deal of speed.  Crucially, this means it is usable on B2G.  Also, it should be possible for people to use it for long-running sessions, which can be helpful for identifying slow leaks.

And while the new version doesn’t currently run on Windows, it should be fairly straightforward to get it working.  If anyone is interested in helping with this, please contact me, or take a look at the open bug.

I will write a more detailed post about the new version some time in the next few days, once I have updated the documentation and finalized a few more tweaks that should improve usability.  In the meantime, Justin Lebar is already using it in earnest to better understand B2G’s memory consumpion (e.g. see here, here, here, here, here, here, here).

Thanks to Mike Hommey for his excellent replace-malloc infrastructure, on which the new version is built, and to Justin for lots of help, especially with Android and B2G details.

B2G

The option that allows B2G to merge system compartments, previously implemented and landed by Kyle Huey, was enabled by default.  This is a big deal, because it’s the single biggest memory consumption improvement B2G has seen and is likely to see before version 1 is released.

Gabriel Svelto reduced the number of unused dirty pages kept around by jemalloc on B2G.  This can save ~4 MiB of memory across all processes, at a potential cost of making some allocations a little slower.  On a device as memory-constrained as the B2G phones, this is a good trade-off.

Social API

Felipe Gomes reduced the memory consumption of the social API when multiple browser windows are open.  This change has been backported to the beta channel, so it will be present in Firefox 18, in time for the bigger publicity push for this feature.  (For those of you that don’t like Facebook, please note that if you don’t enable the social API it will not affect Firefox’s performance in any way.)

Miscellaneous

I added a MEMORY_VSIZE telemetry reporters, which measures virtual memory consumption.  This might help us understand how many Windows users would benefit from 64-bit builds.

Bug counts

Here are the current bug counts.

  • P1: 17 (-3/+3)
  • P2: 126 (-2/+14)
  • P3: 106 (-4/+7)
  • Unprioritized: 0 (-2/+0)
Categories
B2G Firefox Memory consumption MemShrink

MemShrink progress, week 75–76

B2G continues to be a major focus of work relating to memory consumption.  Even the non-B2G-specific improvements made this fortnight were mostly identified and made to help B2G.

B2G-specific

Gabriele Svelto caused dirty freed pages held onto by jemalloc to be purged when a B2G process experiences memory pressure, such as when it goes into the background.  This typically reduces a process’ size by about 2 MiB, so it’s a big deal.  This was a MemShrink P1 bug.

Benoit Jacob prevented the creation of redundant OpenGL contexts on B2G.  This saves 750 KiB in the main B2G process, and 750 KiB in every child that uses WebGL.  Benoit was able to do this based on data from a B2G-specific heap profiler written by Justin Lebar.

I made a large IPC message buffer shrink once all messages from it have been processed.  This saves either 120 KiB or 248 KiB per child process.

James Lal greatly reduced memory consumption of the ical parser during sync.  This change also made it much faster.

Justin Lebar and I added a memory reporter for Freetype, which uses about 2 MiB of memory in the main B2G process.  (This reporter also works on Fennec.)

Andrea Marchesini added a memory reporter for B2G’s gralloc memory.

Miscellaneous

Bill McCloskey made the JavaScript engine clean up more data on memory pressure events.

I increased the size of the chunks used by XPT info’s arena allocator, which saves about 80 KiB per process on 32-bit, and a bit more on 64-bit.

Randell Jesup fixed a PeerConnection leak in WebRTC code.

I added more detail to the JavaScript type inference memory reporters.

Bug counts

Here are the current bug counts.

  • P1: 17 (-4/+1)
  • P2: 114 (-5/+9)
  • P3: 103 (-4/+5)
  • Unprioritized: 2 (-3/+1)
Categories
about:memory add-ons B2G Fennec Firefox Memory consumption MemShrink

MemShrink progress, week 73–74

B2G!  Fennec!  Social API!  They’re all happening, and memory consumption is a big deal for all of them.  Time for a MemShrink report.

B2G

Lots of B2G work is happening, unsurprisingly.  All of these changes have been backported to the Aurora channel.

Kyle Huey landed an important patch that merges system compartments together.  This avoids wasted space caused by having 100s of small compartments.  As Chris Jones said: “It’s a completely different phone with these patches.”  Unfortunately, the preference controlling this behaviour is currently turned off on B2G, because it’s causing some Marionette test failures.  Hopefully they’ll be dealt with soon.  Note that this preference won’t be turned on in desktop builds, and there’s a medium-term plan to avoid this wasted space in a less hacky fashion.

Marco Bonardo disabled Places in B2G, saving about 300 KiB in the main process.

Justin Lebar added code to trigger memory pressure events when lowmemkiller notifications occur.

Justin also tweaked things so that the garbage collector is more likely to run when screenshots are taken.

Jeff Muizelaar fixed a graphics bug that I don’t understand but apparently avoids allocating lots of memory on the B2G unlock screen.

Chris Jones ensured that remote content drop their buffers when they’re hidden.

I tweaked the size of the arena chunks used by XPT, which saved about 120 KiB per process on 64-bit builds, and a bit less on 32-bit builds.

I also shrunk the initial size of the SPS hash table, which saves 48 KiB per process on 64-bit builds and 24 KiB on 32-bit builds.

Memory Reporting

Lots of work happened on the memory reporting front.  This was inspired by  “heap-unclassified” typically being much higher on B2G than on desktop.  Desktop Firefox uses a single process and its memory consumption is usually measured in the 100s of MiBs or more.  In contrast, B2G uses one “main” process and then one process per running app, and the smaller ones are typically around 10 MiB.  As a result, the sundry small per-process things that don’t matter much for desktop loom larger on B2G.  (For the same reason, changes that reduce per-process memory by smallish amounts have outsize value on B2G.)  Most of the following changes have also been backported to the Aurora channel.

Nick Hurley added a “explicit/network/disk-cache” memory reporter.  It typically measures 100s of KiBs in desktop Firefox.  (It doesn’t get used on B2G.)

I added memory reporters “explicit/xpcom/component-manager” and “explicit/xpcom/category-manager”.  Together they measure about 280 KiB per process on 64-bit builds, and a bit less on 32-bit builds.

I added a memory reporter “explicit/script-namespace-manager”.  It measures just over 150 KiB per process on 64-bit builds, and a bit less on 32-bit builds.

I added a memory reporter “explicit/xpcom/effective-TLD-service”.  It measures about 128 KiB per process on 64-bit builds, and a bit less on 32-bit builds.

I added some detail to the JS object memory reporters.  This gives slightly more insight into where this memory is going, but the extra memory measured as a result is usually pretty small.

I fixed the “explicit/atom-tables” memory reporter, which was erroneously always reporting 0 bytes.  It now measures anywhere from a few 100 KiBs to several MiBs of memory.  This was a frustrating bug to find.  DMD exists to check that memory reporters are measuring memory properly, and the reporter was doing its measurements just fine.  But DMD cannot check that the measured amounts are added correctly, and that’s what was going wrong here — a line that should have been this:

return n;

instead was this:

return 0;

In other words, the code I had written was doing the difficult part correctly, but botched the trivial part.  How annoying.

Social API

Felipe Gomes fixed a document leak that occurred when the social sidebar was hidden, and disappeared only when it was unhidden.

Felipe also added code to clear the previous profile when Social is toggled off, which prevents a leak.

The social API still has some significant unresolved memory consumption issues, which are a concern.

Fennec

Kartikaya Gupta (a.k.a. Kats) turned on tab expiration for Fennec.  What this means is that Fennec will now unload the contents of background tabs in certain circumstances — when memory is low, and in some cases when a tab hasn’t been viewed for over an hour — and then reload them when they are brought back into the foreground.  Read Kats’ blog post for more details.

(This change was prompted by Project 256meg, which aims to make Fennec usable on phones with only 256 MiB of RAM.  (Fennec currently requires 512 MiB, according to the official specs.)  Kats has just got to the point where Fennec can actually start on 256 MiB devices.  Although Fennec only uses a single process, there is obviously significant overlap between this goal and B2G’s memory reduction goals.)

An obvious follow-up idea is to add tab expiration to desktop Firefox.  But the bug tracking that idea has seen more heated discussion.  The reason is that tab expiration might cause data loss in some cases.  This is less of a problem on Fennec because (as far as I can tell) there is less of an expectation that in-flight data will be saved on mobile devices.  For example, having processes killed due to memory constraints on mobile is much more common than on desktop.  Still, I expect plenty more arguing before this issue is resolved one way or the other.  One option is to allow it on desktop but have it disabled by default.

Add-ons

Michel Gutierrez fixed a zombie compartment in Video DownloadHelper, which is the 2nd most popular add-on at AMO.   Version 4.9.11 has the fix.

We hit a notable milestone on the add-on front this fortnight:  on November 1st hit bug for tracking known leaks in add-ons — filed 16 months ago — had zero blockers.  In other words, we reached the point where no add-ons were known to leak!  Unfortunately this didn’t last long, and at the time of writing the tracking bug has three blocking bugs.  Still, it’s confirmation that what used to be our biggest memory consumption problem is well under control.

Bug Counts

Here are the current bug counts.

  • P1: 20 (-3/+5)
  • P2: 110 (-10/+7)
  • P3: 102 (-3/+8)
  • Unprioritized: 4 (-2/+4)

As always, the current bug lists can be found by following the links on the MemShrink wiki page.

Categories
add-ons B2G Firefox Memory consumption MemShrink

MemShrink progress, week 71–72

B2G

Justin Lebar led the B2G charge this fortnight, doing the following.

Thinker Li identified that we were using overly large (128 KiB) arena chunks for type inference data in the JS engine.  Reducing them to 32 KiB saved over 3 MiB of memory on B2G.

Fabrice Desré disabled the add-on manager services, saving about 1 MiB.

These changes have all been backported to Aurora, because that’s the current home of the code that will be used for the V1 release of B2G.

Social API

Jared Wein modified the social sidebar so that it unloads itself 10 seconds after it’s been hidden.  This is good because lots of people thought that closing the sidebar was equivalent to disabling the social functionality, and with this change it now effectively is.

Felipe Gomes fixed a major problem which caused the social API to create and leak many unnecessary ports.

Gecko & SpiderMonkey

Benoit Jacob fixed a leak relating to WebGL and cycle collection.  This fixed 10 seconds pauses that occurred after running the RO.ME demo, and also fixed leaks seen in the WebGL version of Nokia Maps.

Kyle Huey fixed a bug that was causing storage code to leak one thread per connection.  This doesn’t affect most Firefox users much, because most connections live for the full lifetime of the process.  However, it could affect some add-ons, and it helped avoid some frequent OOM oranges in some of our tests.

Benjamin Peterson reduced the amount of memory used to compress JavaScript source code.

Add-ons

Zig fixed a zombie compartment in the Visited add-on.

Bug Counts

Here are the current bug counts.

  • P1: 18 (-4/+3)
  • P2: 113 (-2/+14)
  • P3: 97 (-1/+7)
  • Unprioritized: 2 (-13/+2)

Note:  in all my previous MemShrink reports, I mentioned bug fixes as soon as they landed on mozilla-inbound.  As of this week, I’m changing things so that I (mostly) only mention bug fixes that have landed on mozilla-central and thus been marked as RESOLVED.  This will make my life easier (tracking RESOLVED bugs is easy with Bugzilla search), and it will also mean that the changes in bug counts better match the descriptions of bug fixes.  But it means that some bug fixes will be reported two weeks later than they previously would.

Categories
about:memory add-ons B2G DMD Firefox Memory consumption MemShrink

MemShrink Progress, week 69–70

It’s been a bumper two weeks for MemShrink.

B2G

Gregor Wagner tweaked the JS garbage collection heuristics on B2G so that collections occur more frequently.  This prevents too much garbage from accruing, which is important on low-memory B2G devices.

Justin Lebar disabled sync on B2G, because it isn’t used.

Justin also implemented the ability to dump GC/CC logs when a particular signal is received, which is particularly useful for B2G.

B2G’s Operation Slim Fast is in full swing.  Please see the tracking bug for details about what is happening.  While there are a number of important fixes in the pipeline, memory consumption is still a critical issue for B2G simply because the devices don’t have much memory in the first place.  Any additional help from Mozilla developers will be greatly appreciated.  For example, if anyone can help identify unnecessary compartments, that would be helpful.

Memory Reporting

Boris Zbarsky improved the reporting coverage of style sheets.  We’ve seen this improved coverage account for over 1% of explicit on desktop and over 2% of explicit on B2G.

I fixed some double-counting of style sheets, and slightly improved the reporting coverage of style sheets and JS compartments.

Nathan Froyd added memory reporting for telemetry.

Justin Lebar fixed some over-reporting of the JS stack.  Justin also added to about:memory the ability to read JSON memory report data from the clipboard.

I renamed DMD as DMDV, in preparation for its in-progress, non-Valgrind replacement to take over the original name.

Testing With Valgrind

Gary Kwong has done some heroic work recently getting Valgrind tests passing on Tinderbox.  (Strictly speaking, the tests run Memcheck, which is the default Valgrind tool and is synonymous with Valgrind)  His blog post about it is well worth reading.

This is important for MemShrink because Memcheck can detect an important class of memory leaks — ones where all references to a block of memory are lost and so the block leaks forever.  But it’s also important for Firefox as a whole because Memcheck can detect a slew of other memory-related errors, such as writing unaddressable memory (e.g. buffer overruns) and using undefined values in dangerous ways.  These are the kinds of defects that cause crashes, security vulnerabilities, and other major problems.

The tests are currently hidden on TBPL because they only run on mozilla-central once a day, which is generally not enough for a test suite to be reliable enough to be worth unhiding.  Having said that, the ability to write suppressions for errors found with Memcheck complicates considerations;  see bug 800435 for further discussion.

Add-ons

Gregg Lind fixed things so that our Aurora/Beta users are certain to get an up-to-date, non-leaky version of the Test Pilot add-on.  (Version 1.1.2 was the leaky version;  version 1.2.0 or later has the fix.)  This was a longstanding MemShrink:P1 bug.

Erik Vold fixed a zombie compartment in Scriptish.  Version 0.1.8 has the fix.

Josep del Rio fixed a zombie compartment in Speed Dial.  Version 0.9.6.10 has the fix.

Miscellaneous

Boris Zbarsky fixed a bad zombie compartment leak relating to expandos.  This was a recent regression, and the fix is present on Aurora and Beta.  The problem was first reported in the comments of my MemShrink report from four weeks ago — many thanks to Alex for reporting it and preventing it from reaching an official Firefox release!

Bug Counts

Here are the current bug counts.

  • P1: 19 (-2/+5)
  • P2: 101 (-2/+6)
  • P3: 91 (-6/+0)
  • Unprioritized: 13 (-1/+11)

The unprioritized count is high because we spent the first part of today’s MemShrink meeting talking with Gavin Sharp, Jared Wein and Felipe Gomes about the upcoming social API, and then spent some additional time discussing B2G with David Clarke.  As a result we didn’t have time to triage the 23 new MemShrink bugs.  We’ve scheduled an out-of-band MemShrink meeting for next week to get on top of the bug list.

Visiting Mountain View

I’ll be in Mountain View all next week for the JS team work week.  I’ll be happy to meet up with anyone who wants to talk about memory consumption!

Categories
B2G Firefox Memory consumption MemShrink

MemShrink progress, week 67–68

It’s been a busy couple of weeks for MemShrink.

B2G Memory Reporting

about:memory is our main tool for understanding memory consumption. On desktop Firefox it works very well.  On Firefox mobile it works less well, partly because it’s hard to read all that text on a small screen, and partly because you can’t cut and paste, which makes bug reporting difficult.  And on B2G (a.k.a. Firefox OS) it currently doesn’t work at all.

To remedy this, I implemented the ability to import and export memory reporter data in JSON format, and Justin Lebar sped up the exporting of this data and implemented a signal-based mechanism for triggering it.  The idea is that developers can trigger an export of the data from their device and then view it in about:memory in desktop Firefox.

This will allow detailed memory profiling for B2G, which I have listed as #3 on the current MemShrink “big ticket items” list.  The timing is good, because B2G has reached feature-freeze and so focus will now move to its performance and memory consumption as part of “Operation Slim Fast“.

(On a related note, Kartikaya Gupta and other engineers have started looking closely at the memory consumption of Firefox mobile on low-end devices.)

Generic Memory Reporting

I added memory reporting to IonMonkey.  Happily, it appears to use very little memory, especially compared with JaegerMonkey.  This is partly because it generates smaller code than JaegerMonkey, but mostly it’s because IonMonkey is only used to compile the small fraction of code that has been executed many 1000s of times.  In comparison, JaegerMonkey is used to compile the much larger fraction of code that has been executed more than a few times.

Kyle Huey added memory reporters for attribute nodes and attribute maps, which improves the coverage of the memory reporting for the DOM.

Gecko Slimmings

Kyle Huey optimized the representation of repeated inline style rules.  This means that for pages with many elements of the form <foo style="blah">, the repeated style="blah" part is now shared between the elements.  This is a huge win on certain large, auto-generated pages.  For example, on one page that holds a gigantic Mercurial diff, this change reduced 64-bit Firefox’s memory consumption from 1,759 MiB to 1,306 MiB.

Kyle Huey and Olli Pettay fixed a leak relating to web workers.

Patrick McManus fixed a leak in network code that was causing intermittent oranges for Mochitests.

Seth Fowler made nsConsoleService only post a LogMessageRunnable to the event queue if there are any observers for it.  This can save a lot of memory for pages that contains many errors.

Bad Graphics Drivers

An ongoing problem we have is that badly-written graphics drivers can cause outlandish (e.g. multi-GB) memory consumption.  Here’s one example that was fixed by blocklisting one particular AMD driver.  (That driver was also causing lots of crashes.)

It’s got to the point that if someone is seeing excessive memory consumption with no apparent explanation — e.g. immediately after start-up, with few tabs open — I’ll suggest they set layers.acceleration.disabled to true in about:config and there’s a good chance it’ll fix the problem.  (Restarting in safe mode also disables hardware acceleration.)  I don’t have any good ideas on how to improve this situation.

Bug Counts

Here are the current bug counts.

  • P1: 16 (-8/+3)
  • P2: 97 (-2/+10)
  • P3: 97 (-6/+3)
  • Unprioritized: 3 (-3/+3)

In today’s MemShrink meeting we closed and downgraded a number of P1 bugs that (a) have been partially addressed, or (b) are less important than they first seemed, or (c) are duplicates.