MemShrink progress, week 2

Here are some potted highlights from the second week of MemShrink.

  • The beauty of Gregor Wagner’s time-based GC trigger, which I mentioned last week, became ever evident.  It’s been confirmed that nine other bug reports are fixed by this one change.  Great stuff!  See this comment and subsequent comments for an analysis.  Unfortunately, the release drivers decided it wasn’t suitable for back-porting to Firefox 6.  Users who leave their browser open overnight should find Firefox 7 more to their liking, once it’s available.
  • I landed lazy initialization of TraceMonitors and JaegerCompartments, something I wrote about previously.  This gave a 2.89% reduction in the Trace Malloc MaxHeap measurement on CentOS (x86_64) release 5, and probably similar reductions on other platforms.
  • Justin Lebar added hard and soft page fault counts to about:memory.  This makes it clear if Firefox is paging badly.
  • Dão Gottwald fixed a leak in the iQ library used by Panorama.  This bug could cause entire tabs to be leaked in some circumstances.
  • Blake Kaplan avoided the creation of some sandboxes.  This seems to have reduced the number of JavaScript compartments required for techcrunch.com.

There’s less than a week until the development period for Firefox 7 ends.  Hopefully we’ll see some more good MemShrink fixes before that happens.

15 Responses to MemShrink progress, week 2

  1. When I just keep refreshing about:memory page (just keeping f5 pressed), heap-unclassified is the number that fluctuates the most, apparently generating the most garbage.

    Which reporter I should be cheering on? :)

    • Nicholas Nethercote

      sysKin: some JavaScript code is run to generate about:memory. So every time you refresh, you’ll see all the JavaScript numbers change. This includes heap-unclassified, because some JavaScript memory usage isn’t covered by the js/* reporters. You should see a sawtooth pattern: the JS-related numbers will gradually increase, but every so often they’ll drop because a garbage collection will occur. Non-JS numbers (eg. layout, storage, images) shouldn’t change when you refresh.

      (BTW, I see the explicit/js number change faster than heap-classified. But I’m using a trunk build; if you’re running Aurora there are fewer js/* reporters and so more memory ends up in the heap-unclassified bucket.)

  2. Are these improvements available in the nightly channel (or whatever they call it now) ?

  3. Hey Nicholas, from a user: thanks again for driving this. There has been a perception of “Firefox = memory leak” for a long time, and whether or not it is correct, fixing stuff like this helps everyone.

  4. I wish the release drivers would start taking memory issues seriously; it’s clear in their response to Gregor’s patch that they do not. It’s become evident now that many users are still having issues with Firefox 5. The general public really can’t afford to wait for Firefox 7, let alone 6, to fix memory leaks that have been going on since at least 3.5. Obviously this isn’t as big an issue for people using the nightlies or any betas that include the patch, but anyone suffering enormous stability problems under a release build is really unlikely to jump to a beta.

    Considering FF6 is supposed to be a release addressing stability, leaving out a major improvement to memory–Firefox’s biggest stability issue of all–is well beyond shortsighted. So if FF6 is due in seven weeks, and FF7 six weeks after that, that means we can expect improved performance three months from now. Meanwhile, everyone refusing to jump ship from 3.6 (or even 3.5) is in for a bit of an annoyance when they find out that after 6 is out, support for their version will dry up completely.

    At least you’re keeping up the good work and getting in as many fixes as you can while the door is still open. After that, I suppose we’ll have to wait another two or three years before the next round of highly beneficial bug fixes actually gets incorporated into a release.

    • Nicholas Nethercote

      Lummox JR: I’m also frustrated that Gregor’s patch didn’t make FF6. If that patch couldn’t make it, it makes me wonder if any patch relating to memory usage will ever be back-ported to the Aurora channel.

      But at least it will only be a 6 week delay. There won’t be any waits of a year or more.

  5. One of the Firefox interns, Jezreel Ng, did an amazing hack to refresh about:memory a few times a second, with color hilighting to indicate changes. Hopefully he’ll blog about it shortly. But one of the interesting observations was that as it ran one of the items (for JS stuff / strings) would slowly climb due to about:memory overhead, drop at a GC point, and then repeat. Fascinating to watch!

    • Nicholas Nethercote

      Dolske: if you hold down F5 on about:memory you see the same thing :)

      I’ve been talking to Jez, I encouraged him to work on producing some graphs in about:memory that show how memory usage changes over time. I view about:memory as a tool mostly useful for users filing bugs about memory usage, and the red/green highlighting, though it sounds cute, won’t help much there. In contrast, graphs will help a lot — some users have attached graphs from the Windows 7 process monitor thingy, and that can be very helpful.

  6. Masterleep

    I’m writing an HTML5 game using SVG and animating objects with Javascript. Performance in Firefox is generally fine, except there are very frequent stutters during smooth animation. I am ascribing these to GC pauses, but there are lots of them even when I’m not allocating anything directly.

    Are there any memory debugging tools for Firefox that would assist me in understanding this behavior?

    • Nicholas Nethercote

      Masterleep: they could be GC pauses. JavaScript code allocates memory all the time, even if it doesn’t look like it does.

      Go to about:config and set the “javascript.options.mem.log” value to “true”. Then open the error console, and you’ll get a message printed every time a GC happens. That’ll let you know if the stutters are all due to GC.

  7. Just another user here thanking you for the work going into MemShrink. Seems like a lot of progress for two weeks! Looking forward to testing this out on the next train.

  8. Masterleep

    Thanks for the pointer, Nicholas. These are GC pauses, and I don’t think I can do anything about them since my code isn’t allocating anything during the animation loop.

    It’s unfortunate that there’s no way to do smooth animation in Firefox 5, unless I’m missing something. The GC pauses can often be above 100ms on a recent MacBook Pro which is quite noticeable.

    The pauses aren’t too horrible on a freshly started Firefox process, but if you’ve been browsing around for an hour or two, they are bad.

    Demo is here:
    http://dungeoneers.com/bugs/perf/marching-zombies/perf.xhtml

  9. Nicholas Nethercote

    Masterleep: your code will likely be allocating memory even if you’re not obviously creating objects, that’s just how JavaScript works.

    I filed https://bugzilla.mozilla.org/show_bug.cgi?id=668881 for this problem. I guessed that the existing “Masterleep” in bugzilla was you, I hope that’s right :)

  10. Masterleep

    Thank you Nicholas – I will follow up in the bug.