MemShrink progress, week 49-50

System Compartment Reporting

With the recent landing of compartment-per-global, Firefox now regularly has 200+ system compartments at start-up.  However, most of these compartments didn’t have names, which meant that they were merged into a single “[System Principal]” entry in about:memory and about:compartments.

Until last week, that is, when Nils Maier added identifying information to the vast majority of these.  Here’s a small selection of interesting ones from about:compartments on my own machine.

about:blank
about:compartments
about:memory?verbose

chrome://adblockplus-modules/content/FilterClasses.jsm

chrome://browser/content/bookmarks/bookmarksPanel.xul
chrome://browser/content/browser.xul
chrome://browser/content/places/menu.xml
chrome://browser/content/search/search.xml

chrome://chatzilla/content/browserOverlay.xul

chrome://global/content/bindings/button.xml
chrome://global/content/globalOverlay.xul

chrome://treestyletab/content/treestyletab.xul

file:///home/njn/moz/mi0/o64/dist/bin/components/TelemetryPing.js

jar:file:///home/njn/moz/mi0/o64/dist/bin/extensions/uriloader@pdf.js.xpi!/bootstrap.js
jar:file:///home/njn/moz/mi0/o64/dist/bin/extensions/uriloader@pdf.js.xpi!/components/PdfStreamConverter.js

resource:///modules/TelemetryTimestamps.jsm
resource:///modules/sessionstore/DocumentUtils.jsm

resource://gre-resources/hiddenWindow.html

resource://gre/modules/AddonManager.jsm

resource://services-common/preferences.js
resource://services-crypto/WeaveCrypto.js
resource://services-sync/constants.js
resource://services-sync/engines/bookmarks.js

resource://treestyletab-modules/browser.js
resource://treestyletab-modules/lib/animationManager.js

Just from this, it’s obvious that I had about:compartments and about:memory?verbose open at the time.  It’s also obvious that I had the following add-ons installed:  AdBlock Plus, Chatzilla, Tree Style Tab, and pdf.js.  And about:memory now gives at least a partial measurement of how much memory these add-ons are using.  This will help identify add-ons that are using excessive amounts of memory.  (Having said that, I identified the add-on compartments simply by their names.  It’d be great if there was a way to systematically identify them within the code, but I don’t know if that’s possible.)

I also hope people will scrutinize Firefox’s own compartment use closely, and start to file bug reports saying things like “hey, that .jsm module shouldn’t be present, there must be a leak”.  If you want to see what the full list of 200+ looks like, try out a recent Nightly build!

In related news, I also added some new compartment-specific reports, including ones for cross-compartment wrappers.

One consequence of all these change is that the number of entries in about:memory jumped tremendously.  As a result, I aggregated the small entries within each compartment, which reduces the number of entries by a factor of roughly four while still reporting full information for large compartments.  Nils and I also made about:memory more efficient, so the amount of memory required to generate each line dropped by about 20%.  about:memory still takes up memory itself, but it does so at a level that I’m fairly happy with.

Add-ons

For a change, the biggest MemShrink-related news in this report wasn’t related to add-ons!  But there was still some interesting movement there.

Justin Lebar uncovered some evidence that the Hueyfix is having a real, positive effect among users.  Telemetry data from Nightly users shows that the number of ghost windows — a concept for which we don’t have good documentation, but they correlate with zombie compartments — has dropped dramatically, as the following graph shows.

ghost windows telemetry data graph

Telemetry data tends to be extremely noisy, so it’s nice to see a clear signal — Kyle’s change made it into Nightly builds on May 5th [Update: that’s incorrect, see below] and immediately caused the mean number of ghost windows to drop from roughly three to roughly one.  The variance also dropped dramatically.

Update: Justin just wrote a blog post that explains very nicely what ghost windows are.  That post also explains better the circumstances behind the drop in ghost window numbers;  my explanation above was too simple and got the timing wrong.  Thanks, Justin!

In other add-on news, the following add-ons had leaks fixed: Readability, ProxTube, Youtube MP3 Podcaster.

Firefox vs The New York Times

Robert O’Callahan fixed a leak relating to mouse events that triggered when he visited nytimes.com.  He wrote a great blog post explaining the heroic debugging — searching through full memory dumps! — that was required.  It’s great that Robert found and fixed this, though it’s a shame it took such expertise.

Bug Counts

Here are the current bug counts.

  • P1: 23 (-1/+2)
  • P2: 85 (-2/+4)
  • P3: 102 (-5/+2)
  • Unprioritized: 2 (-3/+2)

Not a great deal of movement.  We only had to triage twelve bugs in today’s MemShrink meeting, which is the fewest we’ve had since we switched to fortnightly meetings.

23 Responses to MemShrink progress, week 49-50

  1. Archaeopteryx

    It should be possible to identify the add-ons with compartments.

    You can get the file path with code like this:

    var ioService = Components.classes["@mozilla.org/network/io-service;1"]
    .getService(Components.interfaces.nsIIOService);
    aURL = ioService.newURI("chrome://browser/skin/browser.css", null, null);

    var chromeRegistry = Components.classes["@mozilla.org/chrome/chrome-registry;1"]
    .getService(Components.interfaces.nsIChromeRegistry);
    var newURL = chromeRegistry.convertChromeURL(aURL);
    alert(newURL.spec);

    Then you have to check if it’s a jar file, get the absolute file path of the file, compare with the profile directory [ProfD] and if the file path starts with [ProfD]/extensions/, then it’s an extension and you have to get the remaining part of the url. If it’s a file, get the part left of the dot, else the name of the next folder. Now you have the add-on id and can look it up with


    var addonManager = Components.utils.import("resource://gre/modules/AddonManager.jsm");
    addonManager.getAddonByID(theIDhere)

  2. Archaeopteryx

    Globally installed extensions have a different path, of course.

  3. So, I had a quick peek at about:memory, and three system principal compartments made it onto the summary view, in order of memory:
    about:blank (???)
    resizer.xml (!!!)
    [unnamed system principal compartment]
    Out of the “tiny” compartments, Session Store was the largest.

    By the way, you embedded an insecure link to the telemetry screenshot in your post, so that breaks the lock…

    • Nicholas Nethercote

      How many about:blank compartments did you have? Look for the “[123]” number afterwards.

      How big was resizer.xml’s compartments? It’s only 38KB for me right now.

      I don’t understand your comment about the insecure link. Can you explain some more?

      • I believe he was referring to mixed content breaking your SSL padlock.

        • Nicholas Nethercote

          I used the standard WordPress image uploader to embed the image. And I don’t think this site is meant to be https so I don’t know why the padlock would be present. I’m not an expert on this stuff so maybe I’m missing something…

          • I use HTTPS everywhere, and it makes all subdomains on mozilla.org use SSL – I don’t get a broken padlock here (possibly because its forcing the image to use HTTPS as well).

  4. I’m really enjoying the results from the Hueyfix. It’s not the be-all and end-all but it’s the sort of fix I’ve hoped for over a long period.

    So you’ve got 3 +1 add-ons installed Nick? What a lightweight (not quite a pun but kinda :) ) LOL. I’ve got 17 currently active, still using the same old machine, and Nightly is humming along beautifully, too sweet :) Only problem is that silent updates are very noisy! I keep saying to a colleague at work that he’ll be able to switch back from Chrome soon :)

    Anyway, in about:compartments I’m still seeing the [System Principle] prefix on almost every line, along with a combined 48 anonymous compartments (I think?) at the top, for example:

    [System Principal] [48]
    [System Principal], about:blank [5]
    [System Principal], about:compartments?verbose [2]
    [System Principal], chrome://adblockplus-modules/content/AppIntegration.jsm

    Is that to be expected? In your own example have you just removed the prefix from every line manually? I’m running 15.0a1 (2012-05-29) right now.

    On another note, since Nightly is updated, well, nightly, do you think it’s a limited or skewed source of telemetry data since restarting (upon update) every 24/36 hours (on average?) after an update resets memory usage figures?

    I’m tempted to ignore updates and let this Hueyfixed Nightly run for many days to see how it performs.

    • Nicholas Nethercote

      I removed the “[System Principal]” part by hand. https://bugzilla.mozilla.org/show_bug.cgi?id=759193 is open to remove it or at least slim it down.

      I have 11 anonymous System Principal compartments — Nils annotated most but not all places where compartments can be created. Hopefully the rest will be mopped up in follow-up bugs. I suspect your number is higher due to more add-ons.

      The fact that Nightly builds are restarted could definitely skew their data. Telemetry data is very noisy in general. This seems to be an unavoidable fact of life.

  5. Nils Maier

    I already created an extension lumping together compartments and other identifiable bits by add-on:
    https://github.com/nmaier/about-addons-memory
    See the xpi dir for installable packages.

    That thing has some serious limitations, most importantly: Many add-on use overlays exclusively. The overlay scripts are living in the compartment of the overlaid window, usually browser.xul. Hence the memory these overlay scripts use will appear in the “Nightly” add-on instead of the actual add-on.
    Then again, the counts should be pretty meaningful (not perfect) for most restartless add-ons (not only Jetpacks) or those add-on that use js modules or components for the majority of functionality.

    If you use that, please don’t spam add-on authors with reports like “Adblock Plus uses 25MB” ;)
    Neither is 25MB at much, nor is it over the top if ABP has 50000+ filters loaded in a way that matching is still fast. ;)

  6. B.J. Herbison

    I don’t know what the about:blank compartment measures, but it isn’t about:blank–at least I hope it doesn’t take 8MB of scripts to create a blank page (bug 759581).

  7. There used to be a memory issue of some sort in the addon “Status-4-Evar”, but that was fixed when the author made the following change: “Tear down S4E bindings on window destruction. I don’t know of there being any reported memory leaks, so this is a ‘just in case’ measure.”.

    I am no addon author and thus don’t really have any clue on what this means. Recently however, I have noticed the same kind of memory issue and eventual stuttering/sluggishness with the addon “Reddit Enhancement Suite” as I saw before the fix in “Status-4-Evar”.

    My questions are these: Do anyone know what it means to tear down bindings on window destruction? Also, has the addon “Reddit Enhancement Suite” ever been examined by the MemShrink guys? I’m fearing these two memory issues might be similar, but I need someone with more understanding to comment on it.

    • Nicholas Nethercote

      I don’t know exactly about “tearing down bindings”, but it sounds similar to a lot of the things mentioned in https://developer.mozilla.org/en/Extensions/Common_causes_of_zombie_compartments_in_extensions

      AFAIK no-one from Mozilla has looked at the Reddit Enhancement Suite. I just tried it out briefly and didn’t see any obvious problems like zombie compartments. Can you give steps to reproduce the bad behaviour? Can you get an about:memory snapshot when the bad behaviour is occurring?

      • I don’t know of any specific steps to reproduce the issues with “Reddit Enhancement Suite” other than just to browse reddit a lot.

        The bad behaviour is really just a huge memory usage, which results in some micro freezes that are noticeable when scrolling or playing flash content.

        Firefox will after about 24-48 hours, be using close to 2 GB of memory. I’ve recently restarted Firefox (Nightly), and is currently “only” on 1.2 GB; will try and post a memory snapshot when usage is high again. I should mention that I have many windows/tabs open, 28 according to about:support, but it only use around 6-700 MB on startup.

      • Finally got around to having the browser run a few days, for it to build up to about 2GB of memory usage. Please let me know if there is anything else I can do, to figure out the cause of this issue.

        Skimming over the reports, I’ve noticed a few things:
        * There are in general a lot of URLs of sites I haven’t visited since yesterday.
        * Lot of references to Greasemonkey, is it possible that or one of the two scripts I have installed for it, could be leaking? The two scripts I have are both for YouTube, which might explain all the YouTube URLs found in the reports.
        * The “Reddit Enhancement Suite” addon is “jid1-xufzosoflzsoxg-at-jetpack” also appear many times.

        Reports: Firefox Nightly 15a1 (2012.06.02) – 53 hours:
        * about:memory
        * about:compartments

  8. Hi, a new problem appeared recently, huge memory usage if Jenkins CI screen is just left running in a tab (in the background).

    Same problem in both 12.0 and latest nightly. Just open the browser, with all addons disabled. Then open about:memory and about:compartments, and the local Jenkins 1.464 server main page on the third tab.

    After about 12 hours Firefox uses 1.8GB of memory; 1.4GB explicit, 900MB heap-non-classified, 400MB for the jenkins page compartment.

    Minimize memory usage won’t help. Closing the tab won’t help, the compartment won’t die. No ghost windows on about:compartments.

    This is 100% repeatable, and a bit of a problem also. Is there anything I can do to help track it down?

  9. Hi,

    When I run about:mem I have this warning:

    WARNING: the following values are negative or unreasonably large.
    explicit/storage/sqlite/(14 tiny)
    explicit/storage/sqlite/(14 tiny)/other

    So here is a screenshot of the about:mem could you tell me what is “wrong” since everything seem to run fine ?
    I Use FF13 latest beta under win764 with 16mb ram
    http://screencast.com/t/f8EPFmySIepF

    Regards

  10. My evening browsing habits on my home computer are fairly consistent & I noticed that by the end of the evening, Firefox was always consuming 900-950MB RAM. I figured it was one of my add-ons causing problems as the problem went away when I disabled all of them. Except I refuse to leave my add-ons behind, no matter how much they leak.

    I’d been highly anticipating FF15’s arrival on the Aurora branch with the Chrome-to-Content leak fix. The results were better, but not as good as I was hoping: ~850MB. Something still wasn’t right. But with the new compartment-per-global, I finally tracked down the source… Greasemonkey! One of my self-authored scripts was now creating three duplicate compartments for every normal compartment! I disabled that one script as a test.

    As I prepare to sign off from tonight’s usual browsing routine, Firefox is using just 590MB. Thank you so much for these features. This has been the biggest memory impact for me since the memshrink project started. :D