MemShrink progress, week 51-52

Memory Reporting

Nathan Froyd added much more detail to the DOM and layout memory reporters, as the following example shows.

├──3,268,944 B (03.76%) -- window(http://www.reddit.com/)
│  ├──1,419,280 B (01.63%) -- layout
│  │  ├────403,904 B (00.46%) ── style-sets
│  │  ├────285,856 B (00.33%) -- frames
│  │  │    ├───90,000 B (00.10%) ── nsInlineFrame
│  │  │    ├───72,656 B (00.08%) ── nsBlockFrame
│  │  │    ├───62,496 B (00.07%) ── nsTextFrame
│  │  │    ├───42,672 B (00.05%) ── nsHTMLScrollFrame
│  │  │    └───18,032 B (00.02%) ── sundries
│  │  ├────259,392 B (00.30%) ── pres-shell
│  │  ├────168,480 B (00.19%) ── style-contexts
│  │  ├────160,176 B (00.18%) ── pres-contexts
│  │  ├─────55,424 B (00.06%) ── text-runs
│  │  ├─────45,792 B (00.05%) ── rule-nodes
│  │  └─────40,256 B (00.05%) ── line-boxes
│  ├────986,240 B (01.13%) -- dom
│  │    ├──780,056 B (00.90%) ── element-nodes
│  │    ├──156,184 B (00.18%) ── text-nodes
│  │    ├───39,936 B (00.05%) ── other [2]
│  │    └───10,064 B (00.01%) ── comment-nodes
│  └────863,424 B (00.99%) ── style-sheets

Although these changes slightly improved the coverage of the reporters (i.e. they reduce “heap-unclassified” a bit), the more important effect is that they give greater insight into DOM and layout memory consumption.  [Update:  Nathan blogged about these changes.]

I modified the memory reporter infrastructure and about:memory to allow trees of measurements to be shown in the “Other Measurements” section of about:memory.  The following excerpt shows two sets of measurements that were previously shown in a flat list.

108 (100.0%) -- js-compartments
├──102 (94.44%) ── system
└────6 (05.56%) ── user

3,900,832 B (100.0%) -- window-objects
├──2,047,712 B (52.49%) -- layout
│  ├──1,436,464 B (36.82%) ── style-sets
│  ├────402,056 B (10.31%) ── pres-shell
│  ├────100,440 B (02.57%) ── rule-nodes
│  ├─────62,400 B (01.60%) ── style-contexts
│  ├─────30,464 B (00.78%) ── pres-contexts
│  ├─────10,400 B (00.27%) ── frames
│  ├──────2,816 B (00.07%) ── line-boxes
│  └──────2,672 B (00.07%) ── text-runs
├──1,399,904 B (35.89%) ── style-sheets
└────453,216 B (11.62%) -- dom
     ├──319,648 B (08.19%) ── element-nodes
     ├──123,088 B (03.16%) ── other
     ├────8,880 B (00.23%) ── text-nodes
     ├────1,600 B (00.04%) ── comment-nodes
     └────────0 B (00.00%) ── cdata-nodes

This change improves the presentation of measurements that cross-cut those in the “explicit” tree.  I intend to modify a number of the JS engine memory reports to take advantage of this change.

One interesting bug was this one, where various people were seeing multiple compartments for the same site in about:compartments, as the following excerpt shows.

https://bugzilla.mozilla.org/
https://bugzilla.mozilla.org/, about:blank

It turns out this is not a bug.  The sites in question contain an empty iframe that doesn’t specify a URL, and so the memory reporters are doing exactly the right thing.  Still, a useful case to know about when reading about:compartments.

Add-ons

Justin Lebar fixed the FUEL API, which is used by numerous add-ons including Test Pilot and PDF.js, so that it doesn’t leak most of the objects it creates.  This leak caused 10+ minute shut-down times(!) for one user, so it’s a good one to have fixed.

Kyle Huey briefly broke his own Hueyfix, and then fixed it.  You had us worried for a bit there, Kyle!

asgerklasker fixed a leak in the HttpFox add-on that caused zombie compartments.

Miscellaneous

I restricted the amount of context that is shown in JavaScript error messages.  This fixed a longstanding bug where if you had enabled the javascript.options.strict option in about:config, and you had the error console open or Firebug installed, memory consumption would spike dramatically when viewing pages that contain JavaScript code that triggered many strict warnings.  This bug manifested rarely, but would bring Firefox to its knees when it did.

Asaf Romano fixed a leak that caused a zombie compartment if you used the “Highlight All” checkbox when doing a text search in a page.

Mike Hommey finished importing the new version of jemalloc into the Mozilla codebase, and then wrote about it.  The new version is currently disabled, but we hope to turn it on soon.  Preliminary experiments indicate that the new version is unlikely to affect performance or memory consumption much.  However, it will be good to be on a version that is in sync with the upstream version, instead of having a highly hacked Mozilla-specific version.

Justin Lebar wrote a nice blog post explaining what “ghost windows” are, and how we’ve used them to gauge some recent leak fixes.

Till Schneidereit fixed a garbage collection defect that could cause memory usage to spike in certain cases involving Firefox Sync.

LifeHacker published a new browser performance comparison which looked at Firefox 13, Chrome 19, IE9, and Opera 11.64.  Firefox did the best overall and also won both the memory consumption tests.  I personally take these kinds of comparisons with a huge bucket of salt.  Indeed, quoting from the article:

Our tests aren’t the most scientific on the planet, but they do reflect a relatively accurate view of the kind of experience you’d get from each browser, speed-wise.

If you ask me, the text before the “but” contradicts the rest of the sentence.  What I found more interesting was the distinct lack of “lolwut everyone knows Chrome is faster than Firefox” comments, which I’m used to seeing when Firefox does well in these kinds of articles.

Bug Counts

Here are the current bug counts.

  • P1: 25 (-1/+3)
  • P2: 88 (-3/+6)
  • P3: 106 (-0/+4)
  • Unprioritized: 2 (-2/+2)

Nothing too exciting there.

12 Responses to MemShrink progress, week 51-52

  1. Justin Dolske

    I’ve casually noticed a distinct drop in the omglolfirefoxleaks comments over time. I don’t think it’s unreasonable to attribute much of this to the efforts of memshrink! I look forward to the weekly (or biweekly :) update posts, because there’s awesome payouts that come from the consistent push y’all have made to improve memory usage (and reporting/diagnostics). Kudos!

    • Nicholas Nethercote

      I have noticed such a drop too, and when I do see comments like that now there’s often quite a lot of pushback from other commenters, often asking them if they’ve tried Firefox recently. Encouraging stuff!

      • If you guys are missing such posts, I can easily oblige, LOL :) Actually I’ve been getting a bit bored lately having not much too whine about, hahaha. I’ve been running Nightly at home and work with no substantial trouble. The main thing I can whine about ATM is when Aurora is going to get the Hueyfix. I know there’s that release table but it does my head in trying to interpret it! I want Hueyfix to hit Aurora so I can get the family and friends to run it.

        Actually come to think of it, I finally tried Sync today and it wasn’t an ideal experience. Additionally I really wish Firefox on desktop would get system codec support, especially since the GStreamer SDK was just released, but of course these are not really MemShrink topics at all are they? :)

        If only Snappy reports were as useful as your posts Nic and progress was as visible. I’m not saying Snappy isn’t making progress, just that I find it more difficult to perceive.

        • The “hueyfix” actually is in Aurora now. Though the patch that fixes the thing that temporarily broke it didn’t land there until today, so it won’t be in Aurora until tomorrow. So you may want to hold off a day before recommending people try it out.

          As for Snappy, Taras does make updates about what is being fixed on his blog ( http://blog.mozilla.org/tglek/ ). It is generally a wider-reaching effort by its nature, so it can be a little harder to follow what is going on.

          • Hi

            Thanks for the update on Aurora.

            Yeah I read Taras’ Snappy posts. They just aren’t as clear as Nics :)

  2. Please Ignore This Troll

    >Kyle Huey briefly broke his own Hueyfix, and then fixed it. You had us worried for a bit there, Kyle!

    Doesn’t seem like this patch was ever pushed to aurora repo despite having approval (!).

  3. Am I reading this correctly … Firefox can now report relatively granular memory consumption on a per-site basis such that developers (like me) can justify to managers the time needed to write cleaner DOM code when said managers would rather us hack the fastest code no matter how bad it is?

    I can’t wait to try this on some of our terrible vendorware that I am told to co-opt into something usable.

    This is second only to per-tab and/or per-Add-On memory reporting. If us web devs can see what code produces cleaner, more efficient memory usage, funnily enough we might be able to start writing more of it!

    Hopefully this is the start of something bigger. Developers could finally understand what practices take more memory. For example, does adding excess id and class attributes to elements use more memory (because the browser needs to store a certain amount of information about those references)?

    If I’m not mistaken, if Firefox provides developers with granular memory consumption data, Firefox will be ahead of Google Chrome despite Google’s commitment to speeding up the web through developer support tools like Google PageSpeed.

  4. Is the new ‘magic’ fix for zombie compartments you wrote about some time ago already in ff15 aurora?

    • Yes, the fix for the common problems with addons causing zombie compartments is in Aurora now, though the patch that fixed the accidental breaking of this will only be in Aurora starting tomorrow.

  5. @Nicholas

    I see the graphs rising with new spikes on areweslimyet.com. Are there new features in FF which are the result of that?

  6. you and the others doing a very good job, I’m having two full rows of tabs open and just 300MB allocated and a smooth experience!!