Category Archives: add-ons

MemShrink progress, week 55-56

Memory Reporting

I changed things so that JavaScript memory consumption for web content is reported on a per-tab basis, as the following example shows.

│  ├────4,472,568 B (00.50%) -- top(http://www.mozilla.org/en-US/firefox/fx/, id=236)/active
│  │    ├──4,192,640 B (00.47%) -- window(http://www.mozilla.org/en-US/firefox/fx/)
│  │    │  ├──1,979,112 B (00.22%) ++ js/compartment(http://www.mozilla.org/en-US/firefox/fx/)
│  │    │  ├──1,607,216 B (00.18%) ++ layout
│  │    │  ├────352,520 B (00.04%) ── style-sheets
│  │    │  ├────253,312 B (00.03%) ++ dom
│  │    │  └────────480 B (00.00%) ── property-tables

All the major components of each tab’s memory consumption — JS, layout, style sheets, and DOM — are now reported on a per-tab basis.  This is great, because it’s something that people have been requesting for  years.  However, the presentation is still intimidating.  Consider the following excerpt from about:memory.

├───18,288,696 B (16.43%) -- window-objects
│   ├───6,263,208 B (05.62%) ++ top(about:memory?verbose, id=6)/active
│   ├───5,090,352 B (04.57%) ++ top(chrome://browser/content/browser.xul, id=1)/active
│   ├───3,422,280 B (03.07%) ++ top(http://www.mozilla.org/en-US/firefox/fx/, id=17)/active/window(http://www.mozilla.org/en-US/firefox/fx/)
│   ├───3,178,432 B (02.85%) ++ top(chrome://global/content/console.xul, id=21)/active
│   └─────334,424 B (00.30%) ++ top(resource://gre-resources/hiddenWindow.html, id=3)/active/window(resource://gre-resources/hiddenWindow.html)

Among those top(...) entries are two browser tabs (one for www.mozilla.org, one for about:memory), two browser chrome windows (browser.xul is the main browser window, and console.xul is the error console window), and the hidden window which is always present.  Firefox developers can easily understand this, and technically-inclined users may be able to guess what’s going on, but the average user will struggle.

From a user’s point of view, the visible things they have control over (i.e. windows and tabs) should be better distinguished from everything else, and be shown with nicer names than “browser.xul”.  But the architecture of Firefox’s internals doesn’t make this easy.  I’ve made one not-very-successful attempt to improve this, and I’d love to hear suggestions from other Firefox devs on how to present this information better.

(The step after that one is to present some kind of “tab manager” — a massively stripped-down variant of about:memory that ordinary users can understand.  This would also require localization for all the languages that Firefox supports.  One step at a time!)

Finally, another memory reporting improvement happened:  Andrew McCreight and I added measurement of orphan DOM nodes.  These are DOM nodes that have been discarded by a page but are still accessible from JavaScript objects.   They were the biggest single remaining contributor to “heap-unclassified”.  For example, when I hit the “update” button on about:memory I get 5MB of them (don’t worry, they go away when the garbage collector runs).  Also, badly coded sites can create large numbers of them.  So it’s a good thing to be able to measure, for both users and web developers.

Miscellaneous

Mihai Sucan fixed a bad leak involving pages that use window.console.  Multiple people reported that this caused Firefox’s memory consumption to balloon to multiple GB.  This leak was introduced during the Firefox 15 development cycle, and the fix has been backported to the Aurora channel, which is where Firefox 15 currently resides.  One notable thing about this fix is that the problem was quite easy to diagnose because about:memory clearly indicated that the ConsoleAPI.js compartment was responsible for most of the memory consumption.  Without the fine-grained memory reporting that was enabled by compartment-per-global, that memory would have been reported in a generic “System Principal” compartment which would have made diagnosis much harder.

Wladimir Palant fixed an issue in AdBlock Plus that causes its memory consumption to slowly grow when viewing sites that repeatedly reset the src property of images, such as Mibbit.  (The fine-grained memory reporting enabled by compartment-per-global also made this diagnosis much easier.)  This fix is in Adblock Plus 2.1.

Alexandre Poirot fixed a minor leak in the Add-on SDK.

I wrote about the difficulties of cross-browser memory comparisons.

Bug Counts

Here are the current bug counts.

  • P1: 20 (-3/+0)
  • P2: 90 (-3/+6)
  • P3: 102 (-6/+2)
  • Unprioritized: 2 (-0/+0)

A net reduction of seven bugs!

Update on The big ticket Items

In January, I described six “big ticket items” that needed tackling to improve Firefox’s memory consumption.  Let’s look at how they’ve progressed since then.

#6: Better Script Handling

This had two parts.  The first part was to only generate bytecode for a function once it is run.  I’ve made a good chunk of progress towards and have it working on very simple examples.  This has required a great deal of refactoring of the SpiderMonkey front-end.  (The front-end is a hairy piece of code, so this is virtuous in and of itself.)  There are still a couple of other bugs blocking it from further progress.  I intend to return to continue on this once they are done.

The second part was to share immutable parts of scripts between web pages.  No progress has been made on this.

So, overall, this item is about 25% done, and progress is still ticking along slowly.

#5: Better Memory Reporting

This one also had two parts.  The first part was to reduce about:memory’s “heap-unclassified” number (a.k.a “dark matter”) to typically 10%.  People such as Nathan Froyd and I have made good progress on this.  It’s now occasionally below 10% (my current session shows 7.5%) but often is around 15%.  That’s much better than the 20–25% we typically were getting at the start of the year, and it’s low enough that people rarely complain about it now.  And we’re very much into the long tail now, so it’ll be hard to improve things much further.  For the most part I don’t think we need to.

The second part was to report memory in a per-tab fashion.  As mentioned above, I just completed this, although the presentation is not as user-friendly as it could be.

The per-tab reporting was enabled by compartment-per-global.  Compartment-per-global also gives us much more insight into the memory consumption of Firefox’s own JavaScript code, and even some idea about add-on memory consumption, which was (to me) an unexpected bonus.  Indeed, its usefulness was demonstrated by the two leak fixes mentioned above.

So, overall, this item is about 80% done.  Although there is still some work to be done, it has progressed enough that I will remove it from the “big ticket items” list.

#4: Better Memory Consumption Tracking

We can cross this item off the list, thanks to areweslimyet.com.  In January I also mentioned the idea of using telemetry for this, but we’ve found that telemetry results are so noisy that little (with rare exceptions) can be gleaned from it, so there’s not really anything to be done on that front.

#3: Compacting Generational GC

This is one of the two key SpiderMonkey performance features under development (the other being IonMonkey), and so people such as Terrence Cole, Steve Fink, Brian Hackett, and Bill McCloskey and actively working on it.  Unfortunately, it’s a huge undertaking that requires rewriting many of SpiderMonkey’s internal and external APIs.  If I had to guess I’d say it’s 25% complete, but that really is just a wild guess.  I’ll be pleased if it is finished before the end of the year.

#2: Better Foreground Tab Image Handling

This is the big ticket item for which the least progress has been made. There’s a single bug that is blocking all the follow-on work.  Both Timothy Nikkel and Jet Villegas have spent time working on it, but it has been a difficult nut to crack.  Justin Lebar had a recent suggestion for narrowing its scope slightly.  I hope it’s a good suggestion, because I’m out of ideas on this one.

#1: Better Detection and Notification of Leaky Add-ons

This is the most satisfying item of the lot.  I expected to make only grudging, piecemeal progress, but Kyle Huey’s fix to prevent zombie compartments prevents the vast majority of add-on leaks, enough to declare victory and cross this item off the list.  I’ll be writing more about this next week.

Summary

Three items (#1, #4, #5) have progressed enough that they can be removed from the list.  Only one of these directly improved Firefox’s memory consumption;  the other two were about better measuring and monitoring of memory consumption.

The remaining items are about reducing memory consumption.  Two of them (#3, #6) have a long way to go, but progress has been made and is ongoing.  One item (#2) has made little progress and has stalled.

The New big ticket Items

Here’s my updated list of the most important things.

#5: Better Script Handling

As mentioned above, this has two parts: lazy bytecode generation, and the  sharing of immutable parts of scripts.  I intend to continue working on the first part.  Once that’s done, it may turn out that the second part isn’t necessary;  we’ll see.

#4: Regain compartment-per-global losses

The measurements on areweslimyet.com have been creeping up for the past two months.  The major cause is compartment-per-global, which has lots of benefits (as mentioned above) but also increases memory consumption by a small-to-moderate amount across the board.  This is because there are many more compartments than there used to be, and there’s a certain amount of overhead and waste in each compartment.

Generational GC will help, but there are some other things that can be done more quickly that will help.  See the tracking bug for one example.

#3: Boot2Gecko

Boot2Gecko (now officially known as “Firefox OS”) is going to be running on low-end phones without much physical memory, so that’ll require care.  The first step is to get about:memory working on B2G.  The next step is to find a way to copy the data from about:memory somewhere else, because B2G doesn’t have cut and paste!  Beyond that, it’s unclear;  but once about:memory is working it’s very likely it will identify some sub-optimal B2G-specific behaviour.

#2: Compacting Generational GC

Let me quote myself to explain the memory consumption benefits of compacting, generational GC.

Virtual memory consumption drops in two ways.  First, the compaction minimizes waste due to fragmentation.  Second, the heap grows more slowly.  Physical memory consumption drops for the same two reasons.

The performance benefits are also significant.

Performance improves for three reasons.  First, paging is reduced because of the generational behaviour:  much of the JS engine activity occurs in the nursery, which is small;  in other words, the memory activity is concentrated within a smaller part of the working set.  Second, paging is further reduced because of the compaction: this reduces fragmentation within pages in the tenured heap, reducing the total working set size.  Third, the tenured heap grows more slowly because of the generational behaviour… which means that structure traversals done by the garbage collector (during full-heap collections) and cycle collector are faster.

Important stuff.

#1: Better Foreground Tab Image Handling

On image-heavy pages, Firefox uses much more memory than other browsers.  This includes things like Facebook image slideshows.  One commenter said the following.

On behalf of Facebook Inc, let me just say that we are prepared to give over one thousand free “pokes” to anyone who fixes this bug or alternatively https://bugzilla.mozilla.org/show_bug.cgi?id=683290. I will also gladly show you Mark Zuckerberg’s desk and give you free dinner and cookies as part of a complementary tour of Facebook HQ in Menlo Park.

I have no idea if this person is really from Facebook, but it’s certainly suggestive.  Also, I’ve heard from B2G people that they are having to work around this problem in their Gallery app.

To repeat myself from week 32:  there are three MemShrink:P1 bugs relating to this:  one about not decoding all images immediately, one about discarding non-visible decoded images after some time, and one about some infrastructure work that is required for the first two.  (See this discussion on the dev-platform mailing list for more details about this topic.)

Summary

There are three repeat items, and two new items, but there are fewer items than last time;  this reflects the fact that we are in a better position than we were in January.  If you think I’ve overestimated or underestimated the importance of any of these, or omitted anything, I’d be interested to hear.

One final thing:  last time, several people suggested “better reporting of add-on memory consumption”.  If I knew of any ideas on how to do this better than about:memory currently does, I’d gladly put it on the list.  But I don’t, and there doesn’t seem to be much point to put an item on the list that we don’t know how to implement.

Vacation

I’m taking the next two weeks as vacation, so there won’t be a MemShrink report on July 24.  I’ll be back with the next MemShrink round-up on August 8th.  See you then.

MemShrink progress, week 51-52

Memory Reporting

Nathan Froyd added much more detail to the DOM and layout memory reporters, as the following example shows.

├──3,268,944 B (03.76%) -- window(http://www.reddit.com/)
│  ├──1,419,280 B (01.63%) -- layout
│  │  ├────403,904 B (00.46%) ── style-sets
│  │  ├────285,856 B (00.33%) -- frames
│  │  │    ├───90,000 B (00.10%) ── nsInlineFrame
│  │  │    ├───72,656 B (00.08%) ── nsBlockFrame
│  │  │    ├───62,496 B (00.07%) ── nsTextFrame
│  │  │    ├───42,672 B (00.05%) ── nsHTMLScrollFrame
│  │  │    └───18,032 B (00.02%) ── sundries
│  │  ├────259,392 B (00.30%) ── pres-shell
│  │  ├────168,480 B (00.19%) ── style-contexts
│  │  ├────160,176 B (00.18%) ── pres-contexts
│  │  ├─────55,424 B (00.06%) ── text-runs
│  │  ├─────45,792 B (00.05%) ── rule-nodes
│  │  └─────40,256 B (00.05%) ── line-boxes
│  ├────986,240 B (01.13%) -- dom
│  │    ├──780,056 B (00.90%) ── element-nodes
│  │    ├──156,184 B (00.18%) ── text-nodes
│  │    ├───39,936 B (00.05%) ── other [2]
│  │    └───10,064 B (00.01%) ── comment-nodes
│  └────863,424 B (00.99%) ── style-sheets

Although these changes slightly improved the coverage of the reporters (i.e. they reduce “heap-unclassified” a bit), the more important effect is that they give greater insight into DOM and layout memory consumption.  [Update:  Nathan blogged about these changes.]

I modified the memory reporter infrastructure and about:memory to allow trees of measurements to be shown in the “Other Measurements” section of about:memory.  The following excerpt shows two sets of measurements that were previously shown in a flat list.

108 (100.0%) -- js-compartments
├──102 (94.44%) ── system
└────6 (05.56%) ── user

3,900,832 B (100.0%) -- window-objects
├──2,047,712 B (52.49%) -- layout
│  ├──1,436,464 B (36.82%) ── style-sets
│  ├────402,056 B (10.31%) ── pres-shell
│  ├────100,440 B (02.57%) ── rule-nodes
│  ├─────62,400 B (01.60%) ── style-contexts
│  ├─────30,464 B (00.78%) ── pres-contexts
│  ├─────10,400 B (00.27%) ── frames
│  ├──────2,816 B (00.07%) ── line-boxes
│  └──────2,672 B (00.07%) ── text-runs
├──1,399,904 B (35.89%) ── style-sheets
└────453,216 B (11.62%) -- dom
     ├──319,648 B (08.19%) ── element-nodes
     ├──123,088 B (03.16%) ── other
     ├────8,880 B (00.23%) ── text-nodes
     ├────1,600 B (00.04%) ── comment-nodes
     └────────0 B (00.00%) ── cdata-nodes

This change improves the presentation of measurements that cross-cut those in the “explicit” tree.  I intend to modify a number of the JS engine memory reports to take advantage of this change.

One interesting bug was this one, where various people were seeing multiple compartments for the same site in about:compartments, as the following excerpt shows.

https://bugzilla.mozilla.org/
https://bugzilla.mozilla.org/, about:blank

It turns out this is not a bug.  The sites in question contain an empty iframe that doesn’t specify a URL, and so the memory reporters are doing exactly the right thing.  Still, a useful case to know about when reading about:compartments.

Add-ons

Justin Lebar fixed the FUEL API, which is used by numerous add-ons including Test Pilot and PDF.js, so that it doesn’t leak most of the objects it creates.  This leak caused 10+ minute shut-down times(!) for one user, so it’s a good one to have fixed.

Kyle Huey briefly broke his own Hueyfix, and then fixed it.  You had us worried for a bit there, Kyle!

asgerklasker fixed a leak in the HttpFox add-on that caused zombie compartments.

Miscellaneous

I restricted the amount of context that is shown in JavaScript error messages.  This fixed a longstanding bug where if you had enabled the javascript.options.strict option in about:config, and you had the error console open or Firebug installed, memory consumption would spike dramatically when viewing pages that contain JavaScript code that triggered many strict warnings.  This bug manifested rarely, but would bring Firefox to its knees when it did.

Asaf Romano fixed a leak that caused a zombie compartment if you used the “Highlight All” checkbox when doing a text search in a page.

Mike Hommey finished importing the new version of jemalloc into the Mozilla codebase, and then wrote about it.  The new version is currently disabled, but we hope to turn it on soon.  Preliminary experiments indicate that the new version is unlikely to affect performance or memory consumption much.  However, it will be good to be on a version that is in sync with the upstream version, instead of having a highly hacked Mozilla-specific version.

Justin Lebar wrote a nice blog post explaining what “ghost windows” are, and how we’ve used them to gauge some recent leak fixes.

Till Schneidereit fixed a garbage collection defect that could cause memory usage to spike in certain cases involving Firefox Sync.

LifeHacker published a new browser performance comparison which looked at Firefox 13, Chrome 19, IE9, and Opera 11.64.  Firefox did the best overall and also won both the memory consumption tests.  I personally take these kinds of comparisons with a huge bucket of salt.  Indeed, quoting from the article:

Our tests aren’t the most scientific on the planet, but they do reflect a relatively accurate view of the kind of experience you’d get from each browser, speed-wise.

If you ask me, the text before the “but” contradicts the rest of the sentence.  What I found more interesting was the distinct lack of “lolwut everyone knows Chrome is faster than Firefox” comments, which I’m used to seeing when Firefox does well in these kinds of articles.

Bug Counts

Here are the current bug counts.

  • P1: 25 (-1/+3)
  • P2: 88 (-3/+6)
  • P3: 106 (-0/+4)
  • Unprioritized: 2 (-2/+2)

Nothing too exciting there.

MemShrink progress, week 49-50

System Compartment Reporting

With the recent landing of compartment-per-global, Firefox now regularly has 200+ system compartments at start-up.  However, most of these compartments didn’t have names, which meant that they were merged into a single “[System Principal]” entry in about:memory and about:compartments.

Until last week, that is, when Nils Maier added identifying information to the vast majority of these.  Here’s a small selection of interesting ones from about:compartments on my own machine.

about:blank
about:compartments
about:memory?verbose

chrome://adblockplus-modules/content/FilterClasses.jsm

chrome://browser/content/bookmarks/bookmarksPanel.xul
chrome://browser/content/browser.xul
chrome://browser/content/places/menu.xml
chrome://browser/content/search/search.xml

chrome://chatzilla/content/browserOverlay.xul

chrome://global/content/bindings/button.xml
chrome://global/content/globalOverlay.xul

chrome://treestyletab/content/treestyletab.xul

file:///home/njn/moz/mi0/o64/dist/bin/components/TelemetryPing.js

jar:file:///home/njn/moz/mi0/o64/dist/bin/extensions/uriloader@pdf.js.xpi!/bootstrap.js
jar:file:///home/njn/moz/mi0/o64/dist/bin/extensions/uriloader@pdf.js.xpi!/components/PdfStreamConverter.js

resource:///modules/TelemetryTimestamps.jsm
resource:///modules/sessionstore/DocumentUtils.jsm

resource://gre-resources/hiddenWindow.html

resource://gre/modules/AddonManager.jsm

resource://services-common/preferences.js
resource://services-crypto/WeaveCrypto.js
resource://services-sync/constants.js
resource://services-sync/engines/bookmarks.js

resource://treestyletab-modules/browser.js
resource://treestyletab-modules/lib/animationManager.js

Just from this, it’s obvious that I had about:compartments and about:memory?verbose open at the time.  It’s also obvious that I had the following add-ons installed:  AdBlock Plus, Chatzilla, Tree Style Tab, and pdf.js.  And about:memory now gives at least a partial measurement of how much memory these add-ons are using.  This will help identify add-ons that are using excessive amounts of memory.  (Having said that, I identified the add-on compartments simply by their names.  It’d be great if there was a way to systematically identify them within the code, but I don’t know if that’s possible.)

I also hope people will scrutinize Firefox’s own compartment use closely, and start to file bug reports saying things like “hey, that .jsm module shouldn’t be present, there must be a leak”.  If you want to see what the full list of 200+ looks like, try out a recent Nightly build!

In related news, I also added some new compartment-specific reports, including ones for cross-compartment wrappers.

One consequence of all these change is that the number of entries in about:memory jumped tremendously.  As a result, I aggregated the small entries within each compartment, which reduces the number of entries by a factor of roughly four while still reporting full information for large compartments.  Nils and I also made about:memory more efficient, so the amount of memory required to generate each line dropped by about 20%.  about:memory still takes up memory itself, but it does so at a level that I’m fairly happy with.

Add-ons

For a change, the biggest MemShrink-related news in this report wasn’t related to add-ons!  But there was still some interesting movement there.

Justin Lebar uncovered some evidence that the Hueyfix is having a real, positive effect among users.  Telemetry data from Nightly users shows that the number of ghost windows — a concept for which we don’t have good documentation, but they correlate with zombie compartments — has dropped dramatically, as the following graph shows.

ghost windows telemetry data graph

Telemetry data tends to be extremely noisy, so it’s nice to see a clear signal — Kyle’s change made it into Nightly builds on May 5th [Update: that’s incorrect, see below] and immediately caused the mean number of ghost windows to drop from roughly three to roughly one.  The variance also dropped dramatically.

Update: Justin just wrote a blog post that explains very nicely what ghost windows are.  That post also explains better the circumstances behind the drop in ghost window numbers;  my explanation above was too simple and got the timing wrong.  Thanks, Justin!

In other add-on news, the following add-ons had leaks fixed: Readability, ProxTube, Youtube MP3 Podcaster.

Firefox vs The New York Times

Robert O’Callahan fixed a leak relating to mouse events that triggered when he visited nytimes.com.  He wrote a great blog post explaining the heroic debugging — searching through full memory dumps! — that was required.  It’s great that Robert found and fixed this, though it’s a shame it took such expertise.

Bug Counts

Here are the current bug counts.

  • P1: 23 (-1/+2)
  • P2: 85 (-2/+4)
  • P3: 102 (-5/+2)
  • Unprioritized: 2 (-3/+2)

Not a great deal of movement.  We only had to triage twelve bugs in today’s MemShrink meeting, which is the fewest we’ve had since we switched to fortnightly meetings.

MemShrink progress, week 47–48

Add-ons

The main news in the past two weeks has been about Kyle Huey’s patch that prevents most chrome-to-content leaks, which are the most common kind of add-on leak.  Testing showed it worked beautifully, but caused a knock-on leak in add-ons built with old versions (1.3 and earlier) of the Add-on SDK(This received a lot of attention.)  Kyle then made a slight tweak that fixed that knock-on leak.  So we’re currently still on track for Firefox 15 being “the one that fixes add-on leaks”.

For completeness, here are the add-ons that we know were temporarily affected by that knock-on leak:  Wallflower, Visual Hashing, Translate This!, Easy YouTube Video Downloader.  They (and probably quite a few others) are all working fine again now.

Here’s a quote from an email that one user sent to Kyle this week.

Firefox was leaking about 1.5GB per hour for me. It started with Firefox 3. I tracked it down to Ghostery and NoScript, but even without those addons it leaked about 500MB per hour of browsing.

GC and CC times got up into the 10 second range. Ugly. Really really ugly! And this is on top of the line massively overclocked hardware, too. I had to install a new addon to add a restart button to Firefox, because Firefox froze solid after hitting 2GB of memory usage. I also patched it after every update to allow up to 4GB, buying a little more time…

Then your patch comes along and solves it all… you are awesome man – totally awesome!

Another user — one who uses the leaky Autopager add-oncommented on Kyle’s blog.

Certainly, before this fix I would find that Firefox often became sluggish (input lag, slow paint operations, less than silky smooth scroll animations) as the memory usage built up. It’s hard to say how much various factors contributed to the whole, but GC pauses did undoubtedly cause the scroll animation stuttering.

Restarting was the cure. I haven’t noticed the same symptoms since, and while I haven’t had enough chance to make a conclusive judgement, the signs certainly seem to be good.

I have a full tab strip more often than not, and Fx set to load tabs from last time. This is offset by the wonderful and elegantly simple tabs on demand feature. I’m running a 2 year old laptop with 4GB ram.

And while we’re on the topic, here’s a comment from my blog.

Opened my firefox today, 30+ Tabs (only counting the ones in the active group, the others aren’t loaded), using just little more than 330 MB of RAM. A year ago, with Firefox 4, this would have been impossible. Keep it going!

Good times.

The following add-ons had zombie compartments fixed:  Youtube Ratings Preview, SPDY Indicator  It’s likely these leaks would have been fixed by Kyle’s change, but since Firefox 15 won’t be released until August 28, it’s good that they’ve been fixed now.  (Indeed, the AMO review policy still requires that add-ons not cause zombie compartments with the current release of Firefox;  that policy may be revisited once Firefox 15 is released.)

Compartment-per-global

The other big news is that compartment-per-global (CPG) landed, thanks to the work of various people, especially Luke Wagner and Bobby Holley.  Bobby explained what this means and explored some of the consequences.

CPG will allow lots of things within Firefox to become simpler and faster.  The main disadvantage is, unfortunately, increased memory consumption, as can be seen on areweslimyet.com.  (Thanks to Luke, this increase was less than it could have been.)  This is mostly due to more fragmentation in the JavaScript heap — we now have many more compartments, and each 4KB heap arena cannot be shared between compartments, so there are many more partially empty arenas present.

You might think this would make me bang my head against the wall in frustration, but it doesn’t.  That’s because even if I ignore the many non-MemShrink-related benefits of CPG, there are two big MemShrink-related ones.

First, CPG will enable per-tab memory reporting, something that users have been requesting for years.

Second, CPG will lead to much more detail in about:memory and about:compartments.  For example, Nils Maier has written a patch that makes it obvious all the JavaScript modules that have been loaded.  Another example:  Justin Dolske found that plusone.google.com was doing something silly (constantly creating new iframes?) that caused huge numbers of compartments to be created;  without CPG I think all those globals would have been lumped into a single compartment and the problem would have been much less obvious.  More information in about:memory will lead to more diagnosis of existing problems — particularly leaks of various kinds — in both Firefox and websites.

Memory Reporting

Kevin Locke tweaked the JS memory reporters so that more compartments are distinguished, instead of being lumped together.  This was his first Mozilla patch — well done, Kevin!

Nathan Froyd improved the coverage of the layout memory reporters.  This significantly reduces “heap-unclassified” for huge pages like the single-page version of the HTML5 spec.

Bug counts

Here are the current bug counts.

  • P1: 22 (-2/+3)
  • P2: 83 (-6/+4)
  • P3: 105 (-5/+6)
  • Unprioritized: 3 (-1/+3)

Mostly bouncing around at the moment.

Additional Update on Leaky Add-ons

I wrote last week about leaky add-ons.  Specifically, Kyle Huey landed a patch that in most cases prevents zombie compartments, which are the most common kind of memory leak in add-ons.  However, this change itself caused a different memory leak in some add-ons built with versions 1.3 and earlier of the Add-on SDK.  I described this as “two steps forward, one step back”.

Happily, Kyle landed a patch that slightly delayed the cutting of the chrome-to-content references.  This has the following consequences.

  • It doesn’t compromise the “two steps forward”.
  • It fixes the “one step back”.
  • It should reduce the potential for similar, as-yet-unknown backward steps.
  • The repacking of add-ons with newer versions of the Add-on SDK is now less urgent than it was.  It’s still a good idea, though, because older version had some other, albeit much smaller, leaks.

This is good news.  Firefox 15 is scheduled for release on August 28.  Assuming we don’t hit other problems with these changes prior to release, for users with add-ons there’s a good chance that Firefox 15 will use less memory and suffer fewer annoying pauses.

Once again, it would be great if users of Nightly builds, particularly those that use add-ons, could pay attention to memory consumption and file bugs for any bad behaviour.  Thanks!

Update on Leaky Add-ons

Leaky add-ons are a big problem, but there’s been lots of action relating to them recently.  What follows is a summary of the situation and a request for help from currently Nightly testers who use add-ons.

Two Steps Forward

Kyle Huey’s patch to prevent Chrome-to-Content leaks landed recently.  In theory it would prevent almost all add-ons zombie compartments, which constitute the majority of leaks from add-ons.  And in practice, it appears to be working splendidly.

I did some testing of eight add-ons that are known to leak: McAfee SiteAdvisor 3.4.1(the old version that leaked every content compartment), Firebug 1.9.1, PicPasso 1.0.1, LoL 1.5, YouTube Ratings Preview 1.03, Enter Selects 7, HttpFox 0.8.10, and Link Resolve 0.7.  I tested with a “before” browser build that lacked Kyle’s patch, and an “after” browser build that had it.  For every one of the tested add-ons, I could reproduce the zombie compartment(s) with the “before” build, but could not reproduce any zombie compartment(s) with the “after” build.  In other words, the patch worked perfectly.

To understand how important this is, consider this comment on Kyle’s blog describing the effect of the patch.

I seem to be seeing really significant improvements from this. Normally Fx would start at 170MB and by the end of the day that would often be creeping up to 800 or 900 MB. Today using latest nightly I again started at 170, but now at the end of the day I’m only at 230MB!

That’s a 4x reduction in memory consumption.

What effect does this have?  The obvious benefit of reduced memory consumption is that there will be less potential for paging.  For example, if the above user is on a low-end machine without much RAM, a reduction in memory consumption from 800+MB to 230MB might greatly reduce paging, which would make Firefox much faster.

However, what if the user has a high-end machine with 16GB of RAM?  Then paging isn’t an issue.  But this improvement will still be a big deal on such a machine.  This is because garbage collection and cycle collection cause pauses, and the length of the pauses are roughly proportional to the amount of live heap memory.  (Incremental garbage collection will soon be enabled, which will result in smaller garbage collection pauses, but there are no plans for incremental cycle collection and so cycle collection pauses will still be relevant.)  So even on high-end machines with lots of RAM, leaks can greatly hurt browser performance.  In other words, add-on leaks are a Snappy issue just as much as they are a MemShrink issue.

One Step Back

[Update: Kyle found a simple way to fix this problem with the Add-on SDK.]

So Kyle’s patch is great, but the cutting of the chrome-to-content references was unlikely to be totally free of side-effects.  And sure enough, last week Josh Matthews noticed that Dietrich Ayala’s Wallflower add-on was causing many zombie compartments.  Hmm, wasn’t Kyle’s patch supposed to stop exactly that?  Well, it was expected to stop the most common cases, but there are some other leaks that could still cause zombie compartments.

The weird thing with this leak is that Wallflower is a really simple add-on.  It’s built with the Add-on SDK and is only a few lines of code.  It turns out that Kyle’s patch caused the Add-on SDK to leak badly.  The exact reason is complex;  you can see Kyle’s explanation here.

This sounds really bad.  Lots of add-ons are built with the Add-on SDK.  Have we just exchanged one class of add-on leaks for another?  I don’t believe so, for two reasons.

  1. Kyle’s patch prevents the single most common class of add-on leaks, and the Add-on SDK leak it caused is a much more obscure and unlikely leak.  It’s unfortunate that this obscure leak happened to be in a piece of code that’s used by lots of add-ons.
  2. This obscure leak only occurs in old versions of the Add-on SDK.  The leak doesn’t occur with the latest version, which is 1.6.1.  Indeed, when Dietrich rebuilt Wallflower with version 1.6.1 the problem went away.

Still, the situation is far from ideal, because lots of add-ons are still built with older SDK versions.  So, what to do?  There’s a bug open discussing this question, but it’s gotten bogged down due to differing opinions on what to do.  Version 1.6.1 happened to fix all leaks in the Add-on SDK that were known prior to this problem, so we should be encouraging add-on authors to rebuild with it anyway.

In the meantime, if you use Nightly builds and use add-ons, please monitor about:compartments and file bugs if you find that any add-ons are causing zombie compartments.  If Wallflower is representative, the leaks will be very bad and so shouldn’t be hard to spot.  Firefox 15 is scheduled for release on August 28th;  we need as many affected add-ons to be rebuilt with the latest SDK before that date to minimize potential problems.

Problems with FUEL

Another cause of high memory consumption in add-ons is FUEL.  I know very little about it other than it’s also part of the Add-on SDK sort of a proto-version of the Add-on SDK.  The details are here and here.  This is causing bad performance in Readability and probably other add-ons.  Something to be aware of.

MemShrink progress, week 45–46

FIXING LEAKS

The big news this week is that Kyle Huey made chrome-to-content leaks impossible.  Kyle wrote about this in some detail.  This particular kind of leak can happen in Firefox, but more importantly, it’s responsible for most of the leaks in add-ons that we know about.  In other words, it should greatly reduce the problem of leaky add-ons, a problem I previously identified as the #1 MemShrink issue.  It’ll take a while to see the exact effects and whether they match expectations.  Nonetheless, this is a big deal!

Olli Pettay fixed a leak relating to Geolocation.

The following add-ons had leaks fixed: Collusion, SearchMenu, panchang, HTML5 Extension for Windows Media Player Firefox Plug-in.

Slimmer data structures

I reduced the size of JSScripts (here, here and here).  This increased the number of JSScripts that fit in a GC arena from 31 to 42 (on 32-bit platforms) and from 19 to 28 (on 64-bit platforms).  This reduced the size of the “scripts” entry in about:memory — which is often multiple MB for the compartments of complex web sites and apps like Gmail — by approximately 30%.  (The final two patches in the sequence are being held up by the current tree closure, but will land as soon as that has passed.)

Nathan Froyd reduced the size of mozilla::dom::Link — one of which exists for every <a> or <link> or <area> element shown in a page — by 4 bytes (on 32-bit platforms) and 8 bytes (on 64-bit platforms).

Ehsan Akhgari reduced the size of nsEditor — one of which exists for each text area in a page, and some other places — by 16 bytes on 64-bit platforms.

 Quote of the FORTNIGHT

PC Advisor featured an article about browser performance.  The article criticized the standard browser benchmarks for (a) not representing everyday browsing and (b) underestimating the effects of memory consumption on browser performance — two things that I agree with.

The article also presented some cross-browser memory consumption results from real-world-ish workloads.  The testing was imperfect — for example, it didn’t measure memory consumption after any kind of prolonged usage — but I liked the fact that the author had actually thought about things instead of just mindlessly running the standard benchmarks.

And with my PR hat on, I was happy with the conclusion.

Firefox is the clear winner of the bunch. It was the only browser that did not slow things down and I recommended it for both lower-end mobile devices and high-end desktops.

In general, cross-browser memory consumption comparisons are difficult to do well.  My preferred method is to run some set of benchmarks on a machine with a small amount of RAM, as that gives a genuine indication of the performance effect.  “Browser A was 50% slower on benchmark X when I halved the available RAM, but browser B was only 10% slower” is much more meaningful than “Browser A used 20% more memory on benchmark X than browser B”.  Choosing good benchmarks is still not easy, though.

Bug counts

Here are the current bug counts.

  • P1: 21 (-1/+0)
  • P2: 85 (-14/+15)
  • P3: 104 (-5/+9)
  • Unprioritized: 1 (-0/+1)

I’m thinking about omitting the bug counts from future MemShrink reports.  I don’t feel like they add much, but I’d be interested to hear if people think otherwise.

MemShrink progress, week 43–44

As promised last time, this MemShrink report covers two weeks’ worth of activity.

Melbourne MemShrink Meet-up

This week Kyle Huey, Justin Lebar, Andrew McCreight, Jet Villegas flew to Melbourne to bunker down with me for a MemShrink work week.  This is great for two reasons.  First, it gives all of us an excuse/opportunity to ignore the usual myriad of attention-grabbing tasks and focus solely on MemShrink stuff for a week.  (For example, Jet is the manager of the layout team but he’s managed to do a couple of days of coding!)  Second, it lets us collaborate and pick each others’ brains much more easily and efficiently than normally.

Badd-ons

The following add-ons had zombie compartments fixed.

I’ve said before that add-on leaks are the #1 cause of high memory consumption in Firefox.  Numerous add-ons have had leaks fixed, but this is a slow process.  However, Kyle has been making good progress this week on a patch that has the potential to mitigate the majority of add-on leaks that cause zombie compartments.  It currently passes all tests on try server and is undergoing review.  Watch this space!

Version 1.6 of the Add-on SDK was released, which fixed all remaining known leaks.  Except that a new leak was found shortly before release!  This was found too late to be fixed before the deadline for 1.6, but fortunately, it was quickly fixed in version 1.6.1.  The Add-on SDK developers (especially Alexandre Poirot) have fixed a ton of leaks in the past few months, so it’s great to see this milestone reached.

Miscellaneous

Olli Pettay fixed a leak relating to event listeners.

Henrik Skupin released version 0.3 of the MemChaser add-on.  It features improvements to the UI and the logging.

Jonathan Kew optimized the representation of font character maps.  Depending on the platform, this can save around 1MB of memory, or more if you have lots of fonts installed.

Jared Wein landed an option for making plugin content click-to-play.  This isn’t marked as a MemShrink bug but if a user can ignore such content, it has the potential to reduce memory consumption significantly in some cases.

Bug counts

Here are the current bug counts.

  • P1: 22 (-1/+2)
  • P2: 84 (-56/+3)
  • P3: 100 (-8/+18)
  • Unprioritized: 0 (-1/+0)

Lots of movement there.  This is because we’ve been going through the P2 bugs and closed a lot of them because they were dups, or lacked enough information to do anything useful, or we decided they were no longer worthwhile.  We also downgraded quite a few P2 bugs to P3.  As a result, the P2 list is much more interesting than it was.  And we’re only partway through it!

A new meeting schedule

I mentioned last time that I would be writing these reports every two weeks from now on, instead of weekly.  Similarly, we’ve decided to reduce the frequency of MemShrink meetings to once every two weeks.  The next meeting will be on May 1st.

This change does not reflect a reduction in the amount of work being done on MemShrink bugs.  Rather, it reflects a reduction in the number of new MemShrink bugs being filed.  At the past few meetings we’ve had only a handful of new bugs each time and triage has only taken a few minutes.  (In comparison, in the early days of MemShrink, several times we didn’t get through all the new bugs in an hour.)

This leaves us lots of time for talking about other things;  in fact, more time than we’ve needed.  And time spent in meetings is time not spent fixing problems, so this feels like the right thing to do.

MemShrink progress, week 42

 Are we slim yet?

John’s Schoenick’s areweslimyet.com (AWSY) has had its password removed and is now open to the public!

areweslimyet.com screenshot

This is a major milestone for the MemShrink project.  It shows the progress we have made (MemShrink started in earnest in June 2011) and will let us identify regressions more easily.

John’s done a wonderful job implementing the site.  It has lots of functionality:  there are multiple graphs, you can zoom in on parts of graphs to see more detail, and you can see revisions, dates and about:memory snapshots for individual runs.

John has also put in a great deal of work refining the methodology to the point where we believe it provides a reasonable facsimile of real-world browsing;  please read the FAQ to understand exactly what is being measured.  Many thanks also to Dave Hunt and the QA team for their work on the Mozmill Endurance Tests, which are at the core of AWSY’s testing.

Update: Hacker News has reported on this.

Ghost windows

Frequent readers of this blog will be familiar with zombie compartments, which are JavaScript compartments that have leaked, due to defects in Firefox or add-ons.  Windows (i.e. window objects) can also be leaked, and often defects that cause compartments leaks will cause window leaks as well.

Justin Lebar has introduced the notion of “ghost windows”.  A ghost window is one that meets the following criteria.

  1. Shows up in about:memory under “window-objects/top(none)”.
  2. Does not share a domain name with any window under “window-objects/active.
  3. Has met criteria (1) and (2) for a moderate amount of time (e.g. two minutes).

The basic idea is that a ghost window has a high chance of representing a genuine leak, and this automated identification of suspicious windows will make leak detection simpler.  Justin has added ghost window tracking to about:memory, about:compartments, and telemetry.  (These three bugs were all marked as MemShrink:P1.)  Ghost window tracking is mostly untested right now, but hopefully it will become another powerful tool for finding memory leaks.

Add-ons

We’ve been tracking leaky add-ons in Bugzilla for a while now, but we’ve never had a good product/component to put them in.  David Lawrence, Byron Jones, Stormy Peters and I have together created a new “Add-ons” component under the “Tech Evangelism” product.  The rationale for putting it under “Tech Evangelism” is that it nicely matches the existing meaning of that phrase — it’s a case where a non-Mozilla entity is writing defective code that interacts with Firefox and hurts users’ experiences with and perceptions of Firefox, and Mozilla can only inform, educate and encourage fixes in that defective code.  This component is only intended for certain classes of common defects (such as leaks) that Mozilla contributors are tracking.  It is not intended for vanilla add-on bugs;  as now, they should be reported through whatever bug-reporting mechanism each add-on uses.  I’ve updated the existing open bugs that track leaky add-ons to use this new component.

Leaks in the following add-ons were fixed:  Video DownloadHelper (the 2nd most popular add-on on AMO!), Scrapbook Plus, Amazon Price Tracker.

Bug counts

This week’s bug counts:

  • P1: 21 (-5/+0)
  • P2: 137 (-1/+7)
  • P3: 90 (-1/+3)
  • Unprioritized: 1 (-1/+1)

Good progress on the P1 bugs!

A new reporting schedule

Many of the weekly MemShrink reports lately have been brief.  From now on I plan to write a report every two weeks.  This will make things easier for me and will also ensure each report is packed full of interesting things.  See you again in two weeks!

MemShrink progress, week 41

Add-ons

Jorge Villalobos encouraged add-on authors to check for memory leaks on the official Mozilla add-ons blog.

The following add-ons had memory leaks fixed: FavSync, Download Youtube Videos, Page Speed.

The follow add-ons were downgraded to “preliminary review” status on AMO, giving them less visibility: QuickDrag, Yahoo Toolbar.  The Yahoo Toolbar case is interesting;  it is available on AMO, but only has 47,618 users there.  But the number of installed copies is much larger than this.  (I can’t say how much larger, because the exact numbers are not public knowledge.)  So it’s clear that most people who have it installed got it elsewhere, probably directly from Yahoo.  Given that AMO accounts for only a tiny fraction of the installations, the effect of the downgrade in AMO review status is negligible.  Furthermore, Jorge’s attempts to contact Yahoo appear to have failed.  Finally, the only other means we have for addressing add-ons with problems is block-listing them, but that would be overkill in this case.  So we are at an impasse.  It’s a frustrating situation!

Gecko and Firefox

Tim Taubert fixed two leaks in the recently updated new tabs page.

Jonathan Kew added memory reporters for thebes font objects.  With text-heavy pages open, these can account for 10s of MBs of memory.

Andrew McCreight fixed some shutdown leaks involving the XBL cache.

Igor Bukanov did something with JS principals that I won’t pretend to understand.  This bug wasn’t marked with the MemShrink tag, but it gave good Talos results, reducing the number of malloc calls and the maximum heap size on several platforms.

Bug counts

This week’s bug counts:

  • P1: 26 (-2/+4)
  • P2: 131 (-1/+3)
  • P3: 88 (-5/+1)
  • Unprioritized: 0 (-0/+1)

Nothing spectacular there.