Update on Leaky Add-ons

Leaky add-ons are a big problem, but there’s been lots of action relating to them recently.  What follows is a summary of the situation and a request for help from currently Nightly testers who use add-ons.

Two Steps Forward

Kyle Huey’s patch to prevent Chrome-to-Content leaks landed recently.  In theory it would prevent almost all add-ons zombie compartments, which constitute the majority of leaks from add-ons.  And in practice, it appears to be working splendidly.

I did some testing of eight add-ons that are known to leak: McAfee SiteAdvisor 3.4.1(the old version that leaked every content compartment), Firebug 1.9.1, PicPasso 1.0.1, LoL 1.5, YouTube Ratings Preview 1.03, Enter Selects 7, HttpFox 0.8.10, and Link Resolve 0.7.  I tested with a “before” browser build that lacked Kyle’s patch, and an “after” browser build that had it.  For every one of the tested add-ons, I could reproduce the zombie compartment(s) with the “before” build, but could not reproduce any zombie compartment(s) with the “after” build.  In other words, the patch worked perfectly.

To understand how important this is, consider this comment on Kyle’s blog describing the effect of the patch.

I seem to be seeing really significant improvements from this. Normally Fx would start at 170MB and by the end of the day that would often be creeping up to 800 or 900 MB. Today using latest nightly I again started at 170, but now at the end of the day I’m only at 230MB!

That’s a 4x reduction in memory consumption.

What effect does this have?  The obvious benefit of reduced memory consumption is that there will be less potential for paging.  For example, if the above user is on a low-end machine without much RAM, a reduction in memory consumption from 800+MB to 230MB might greatly reduce paging, which would make Firefox much faster.

However, what if the user has a high-end machine with 16GB of RAM?  Then paging isn’t an issue.  But this improvement will still be a big deal on such a machine.  This is because garbage collection and cycle collection cause pauses, and the length of the pauses are roughly proportional to the amount of live heap memory.  (Incremental garbage collection will soon be enabled, which will result in smaller garbage collection pauses, but there are no plans for incremental cycle collection and so cycle collection pauses will still be relevant.)  So even on high-end machines with lots of RAM, leaks can greatly hurt browser performance.  In other words, add-on leaks are a Snappy issue just as much as they are a MemShrink issue.

One Step Back

[Update: Kyle found a simple way to fix this problem with the Add-on SDK.]

So Kyle’s patch is great, but the cutting of the chrome-to-content references was unlikely to be totally free of side-effects.  And sure enough, last week Josh Matthews noticed that Dietrich Ayala’s Wallflower add-on was causing many zombie compartments.  Hmm, wasn’t Kyle’s patch supposed to stop exactly that?  Well, it was expected to stop the most common cases, but there are some other leaks that could still cause zombie compartments.

The weird thing with this leak is that Wallflower is a really simple add-on.  It’s built with the Add-on SDK and is only a few lines of code.  It turns out that Kyle’s patch caused the Add-on SDK to leak badly.  The exact reason is complex;  you can see Kyle’s explanation here.

This sounds really bad.  Lots of add-ons are built with the Add-on SDK.  Have we just exchanged one class of add-on leaks for another?  I don’t believe so, for two reasons.

  1. Kyle’s patch prevents the single most common class of add-on leaks, and the Add-on SDK leak it caused is a much more obscure and unlikely leak.  It’s unfortunate that this obscure leak happened to be in a piece of code that’s used by lots of add-ons.
  2. This obscure leak only occurs in old versions of the Add-on SDK.  The leak doesn’t occur with the latest version, which is 1.6.1.  Indeed, when Dietrich rebuilt Wallflower with version 1.6.1 the problem went away.

Still, the situation is far from ideal, because lots of add-ons are still built with older SDK versions.  So, what to do?  There’s a bug open discussing this question, but it’s gotten bogged down due to differing opinions on what to do.  Version 1.6.1 happened to fix all leaks in the Add-on SDK that were known prior to this problem, so we should be encouraging add-on authors to rebuild with it anyway.

In the meantime, if you use Nightly builds and use add-ons, please monitor about:compartments and file bugs if you find that any add-ons are causing zombie compartments.  If Wallflower is representative, the leaks will be very bad and so shouldn’t be hard to spot.  Firefox 15 is scheduled for release on August 28th;  we need as many affected add-ons to be rebuilt with the latest SDK before that date to minimize potential problems.

Problems with FUEL

Another cause of high memory consumption in add-ons is FUEL.  I know very little about it other than it’s also part of the Add-on SDK sort of a proto-version of the Add-on SDK.  The details are here and here.  This is causing bad performance in Readability and probably other add-ons.  Something to be aware of.

62 Responses to Update on Leaky Add-ons

  1. My Firefox used to use more and more constant cpu time the more tabs I opened and the longer it ran. Easily reaching 60% cpu time on a 3Ghz Core2. Current versions just drop to 0%! I love that.

    • Nicholas Nethercote

      Can you be more specific about “used to”? When did you see the improvement? Do you use many add-ons?

      • Firefox 4 was really bad with this. I only use adblock plus, ghostery and noscript.

      • I think it started to get solved with all the memory improvements in v7. But overall memory usage and cpu usage have been improving since V6.

  2. Which version of Wallflower is the version rebuilt using the 1.6.1 SDK?

    • According to [1] it’s the current stable version (1.1) on AMO.

      [1] https://addons.mozilla.org/en-US/firefox/addon/wallflower-1/versions/

      • Thanks Erunno. That’s the version currently listed in my Add-Ons page: 1.1. I guess it must have silently upgraded. Still getting familiar with the silent upgrade thing :) Kudos to silent upgrading I guess otherwise my switch to Nightly to take advantage of Huey’s patch might have resulted in more frustration if I was still using the old version of Wallflower and this ‘one step back’ issue had hit me.

  3. Checked about:compartments, see some links listed in “Ghost Windows” section, is this indication of some leak?

  4. We aren’t using Fuel in Addon SDK. But we have similar practices: we create high level wrappers for windows, tabs, … but we are destroying them on window/tab close or addon disabling.
    You linked bugs related to Fuel. Is there opened bugs about “fuel-like” or fuel-related leaks happening with SDK addons?

  5. Ver Greeneyes

    I was wondering, have you considered tracking a profile of Firefox with (a selection of) the most popular extensions and plugins installed on areweslimyet.com? I realize this might not be easy to do in practice – presumably at least some of the most popular addons have very similar and overlapping goals, and wouldn’t make sense to be installed together. But putting together such a profile (which would be somewhat overloaded with addons, but that’s the point) might make an interesting extra metric.

    • Hi Ver,

      I had left this similar question on a previous post:

      “Hi Nick,
      I had a question – is it possible to perhaps have areweslimyet.com but with the top 100 addons added to the test builds?

      So we can see the combined improvements both from the addon authors and from Firefox itself, in single graph?

      I feel this would represent much more of a real working case graph.”

      I do think this idea would be a good way to see how memory consumption with todays build + latest addons, vs a 6 month old build with 6 month old addons compares.

      I would be happy to create a profile with the top 100 addons installed if man-powers an issue?

      • Nicholas Nethercote

        “I feel this would represent much more of a real working case graph”

        I suspect there are exactly zero Firefox users who have the top 100 add-ons installed. Even if we restricted it to, say, 10 add-ons, I’m not convinced this would tell us anything useful.

        • Sequoia McDowell

          Would the addons’ performance not be measured one by one? I don’t have the top 100 addons but I probably have a few of them, and if one is a particular problem I’d like to know. I’ve often yearned for an addon profiler (like windows task manager or unix top) to see what addon in particular is using how much memory.

          • They’re still working on getting memory-per-tab, and memory-per-addon would be a lot more complicated.

  6. Would it be possible to search through AMO addon code looking for code that invokes this “obscure and unlikely leak”?
    If possible, these addons could be flagged/warned.
    The code involved may be too complex to find in such a manner, but it was a thought.

  7. Nicholas, from these two which is under investigation:
    JetPack addons:
    https://addons.mozilla.org/en-US/firefox/tag/jetpack

    All restarless (since it was previously only possible with Addon SDK if not wrong)
    https://addons.mozilla.org/en-US/firefox/tag/restartless

    Add in article which is prime suspect so users who want to test can easily grab list and start testing even if they test only one.

  8. my firefox 12 on Lion start with about 500Mb and at the end of the day reach around 2Gb

  9. i know it, I want just to give my “general user experience” on “firefox” and “memory usage”,

    • Fair enough. How much RAM does your system have? 2GB is not necessarily a high figure if your system has 4GB or more.

  10. If you’re removing junk, it makes me wonder why it was even added to Firefox to begin with. Keep that in mind when adding features. It’s supposed to be a good browser, nothing more.

    Or at least release Firefox Bloatware Edition and Firefox Sane Edition, and let people decide for them selfs.

  11. Running Nightly 15.0a1 (2012-05-07) I’m experiencing what seems strange. The Hueyfix seems to be working well however last night I checked about:compartments and there were a grand total of zero Ghosted Windows.

    About 24 hours later there are over 70 lines under Ghosted Windows, 89 lines under User Compartments and 144 in System Compartments (61 have a memchaser resource URI). Many of the lines under both Ghosted Windows and User Compartments refer to sites I have long since closed.

    However memory used is only a modest 310 MB. This is acceptably low in my experience. I have 23 tabs loaded (approximately 5 in cache) and 26 extensions installed.

    Can anyone shed any light on this situation? After the Hueyfix, do compartments get ‘disconnected’ (for want of a better term) in a memory usage sense, but stay listed in about:compartments?

    I did install the au-revoir-utm 0.1.3 add-on overnight which is in the list of leaky SDK Add-Ons however none of my other Add-Ons are in that list. Would installing that Add-On explain all the Ghosted Windows and closed sites appearing in User Compartments?

    • Hmmm, to partly answer my own question, I disabled au-revoir-utm (however I did not restart) and now all Ghosted Windows lines have vanished. Does that make sense? Or did I perhaps just expect Ghosted Windows to be cleared quicker than they ordinarily would?

      User Compartments lines are still present though and have grown by one.

      • Yes, the SDK leak in Nightly behave exactly like this.
        A lot of User compartments are leaked and only cleared on addon disabling.

        You may give another try after when following bugs are going to be fixed:
        https://bugzilla.mozilla.org/show_bug.cgi?id=752877
        https://bugzilla.mozilla.org/show_bug.cgi?id=751621
        I’m expected them to fix this SDK leak.

      • Nicholas Nethercote

        Just having one add-on from that list would be enough to cause lots of problems. I suspect/hope if you restart with au-revoir-utm disabled you’ll not get any zombie user compartments.

        Having lots of system compartments open seems reasonable when you have lots of add-ons installed.

        As for ghost windows, they’re new enough that I’m not quite sure how to interpret them.

        • Thanks ochameau and Nick.

          I restarted tonight after installing the latest Nightly build – with au-revoir-utm disabled. User Comparments are both a reasonable number and generally legitimate again. Good news there and also zero Ghosted Windows.

          Glad I’m not the only MemShrink-interested person who doesn’t quite know what Ghosted Windows mean. Thought I’d missed a post or something :)

          Is 26 active add-ons still considered ‘lots’ according to telemetry data which I presume counts such stats? I forget what the average number of add-ons installed is though I expect it’s been mentioned in this blog before.

          I’m curious if the mozilla developer community feels that not all add-ons are made equal, so to speak.

          A user might have several add-ons that are relatively ‘inert’ in that they seemingly do nothing from a front-end view until they are specifically activated. E.g., in my case: Fireshot; Sage; Delicious’s ‘Tag’ bookmarking function; TinyURL Generator etc.

          Then there’s the more ‘active’ page or URL modifying add-ons. In my case Adblock Plus, it’s El Hiding Helper; ShareMeNot; Stylish; NoScript; NoSquint and others.

          If users install more than a handful of ‘active’ add-ons, they shouldn’t expect pages to render with the same performance as if these active add-ons were absent. However I think there should be a distinction between active and relatively inert add-ons before a user is considered to have too many add-ons for their own good.

          In my case I have around 6-8 ‘active’ add-ons (ShareMeNot, Textarea Cache, NoScript, NoSquint, Adblock Plus, Stylish, Wallflower) and 7 ‘inert’ add-ons (Delicious, Sage; TinyURL Generator; FEBE; Download Manager Tweak; Vacuum Places Improved; Firebug). I guess a third and last category is global browser UI and behavior modification. Some of these might be borderline ‘active’, I’m not sure (Extended Statusbar; Status-4-Evar; Context Search; Tab Mix Plus; Toolbar Buttons; Old Default Image Style).

          So you can start to see where my 26 active add-ons come from. A lot of them re-instate features otherwise still active in the browser but removed by the almighty minimalist, form-over-function gods. About as many implement functionality Firefox could easily build in but instead chooses to leave to the realm of add-ons.

          Re many system compartments, I could relate to your suggestion that these are likey with more add-ons installed, if I could see at least one Sys Compartment per add-on. However only 5 contains URI referencing unique add-ons. Another 39 have this URI in common:

          resource://jid1-ub4sjepvr2m4qq-at-jetpack/*

          and another 60 lines contain this URI:

          resource://memchaser-at-quality-dot-mozilla-dot-org/*

          Add the files under ..-at-jetpack appear to be legit add-on libraries. Should MemChaser have so many System Compartments?

          Lastly there’s

          about:blank [37]

          in the User Compartments. Wasn’t there some discourse about dead about:blank pages staying in memory a while back? Should any of those 37 be there? Perhaps it’s a result of my well worn profile?

          Keep up the great work MemShrinkers!

          • Nicholas Nethercote

            Justin has promised some documentation on ghost windows next week.

            26 add-ons is still a lot, putting you in the top 0.1% or something. I don’t know of any official Mozilla stance, but I personally think it’s unusual but not unreasonable. Having said that, if you install more add-ons, it’s unavoidable that you are more likely to have problems. Just think: you’re almost certainly running a unique combination of add-ons, which means your Firefox configuration is also unique, and thus untested.

            As for your active/inert categorization, one thing we’ve learnt is that it’s not possible to make this kind of categorization reliably, at least in terms of effects on Firefox performance. Some seemingly innocuous add-ons have had bad memory leaks. Some invasive add-ons are well-written (e.g. ABP) and have relatively small performance effect.

            On the issue of what should be core functionality and what should be provided by add-ons, no matter what Mozilla does someone will complain. It sounds like you don’t like the minimalism, but I’ve lost count of the number of times I’ve seen people say the opposite — “Firefox is bloated, they should go back to the original philosophy of a basic browser and use add-ons for extended functionality”. One person’s bloat is another person’s can’t-live-without-it feature.

            I don’t know why MemChaser causes so many system compartments.

            I’ve seen about:blank compartments like that too, though not that many. It seems to be related to the presence of add-ons. E.g. I had MemChaser installed for a while and typically had about 3 of them, and now that I’ve disabled MemChaser I’m down to usually 1. I don’t know what the cause is.

            (ps: I appreciate the more civil tone you’ve been using lately! Thanks.)

          • Hi Nick

            Thanks for the response. It’s ironic that you appreciate my tone when really this evening I could perhaps rightfully get fairly annoyed again. Alas just when I really thought that MemShrink was the on verge of completely reforming Firefox’s memory consumption, I got home just now and Nightly’s memory all but killed my system again by growing seemingly beyond any memory pressure event GC kickin to 949MB on a 1GB system with shared video memory.

            I really wish I knew what causes this. For so long I’ve been told it’s too many add-ons and various other excuses. *sigh* It’s very disappointing to say the least.

            I suspect the only thing left to blame is a dirty profile. Wasn’t there a tool on the horizon that could migrate data from one profile to another and thus remove any corruptions or outdated structures in old profiles from the picture? I think that is a very good idea and perhaps critical to retaining users who do not want to lose all their settings since it’s such a pain to get them back. I noticed recently that migrators from Chrome and/or Safari to Firefox were added. Perhaps the next migrator should be from an old Firefox profile to new one? Firefox to Firefox?

            I don’t use Sync. Maybe if I used that as a migration tool between profiles it might serve the same purpose until such a tool exists? I wonder if there’s any documentation on using Synch this way?

            On the question of how many add-ons is too many, at the risk of being censored for being critical, I really think it’s hypocritical of Mozilla to keep pushing the implication that a very core differentiating feature of the browser – add-ons – since day one, should not be embraced beyond n add-ons. At the very least this should not be implied but official. It is dishonest to keep marketing a product by promoting a feature beyond it’s effectiveness. I understand this might be a difficult line to draw but I think it should be attempted.

            The Hueyfix is the most quintessential evidence that Mozilla has been promoting add-ons but then denying them without actually taking responsibility for them. Whilst I realize not everything can be prioritized into an ideal timeframe, Firefox’s add-ons have been promoted 8 years or more and the true core of the performance problems with them has just been (we hope) found now. *sigh*

            Until events of the last 12 hours (memory usage ballooning again) I was experiencing what Firefox should be: low memory usage without heavy (IMHO) restrictions on add-ons. I understand that there will always be poorly written add-ons but poorly written should not mean OS-corrupting. Maybe some people would say that’s a pipe dream but I’d like to think that Mozilla can truly extend it’s mandate beyond just providing a vehicle to the open web, to also helping people to use the open web more effectively. This could take the form of using it’s testing infrastructure to test recalcitrant setups like mine. To me that is ‘software as a service’ rather than the different meaning that phrase is given in general IT circles. Don’t get me wrong. If there’s a dodgy add-ons I’ll kill it. I believe I take a fairly active role in maintaining my software as well. I don’t expect a computer to be completely set and forget, so to speak, or for Mozilla to massage thousands or millions of setups worldwide but a config testing and validation service using your try servers for people having repeated problems doesn’t sound unreasonable. Of course all the unique hardware scenarios couldn’t be replicated but couldn’t a user’s Firefox setup on X platform with Y patches be replicated centrally and tested? In a way this is already happening in the opposite direction when users volunteer to test their own systems at our end for WebGL compliance, for example. Why not the other direction?

            I hope the above doesn’t sound too crazy. Futuristic or idealistic maybe, but not too crazy :)

            Regarding the question of native vs add-on features, I think this is really very badly misinterpreted. In most cases I suspect users who say “Firefox is bloated” are simply expressing what I have for years: the MemShrink issue. I know nobody seems to want to admit it, but all the bloat MemShrink has addressed so far would ideally have been dealt with a long time ago. Instead I suspect users now think Firefox is bloated in how it performs, not how it looks visually, which until MemShrink achieves it’s goals and regressions are vigilantly hunted and killed, is a fair assessment IMHO. Firefox 12 is still bloated compared to Nightlies with the Hueyfix (notwithstanding my latest balloon memory experience). However this ‘Firefox is bloated” is being used to justify interface and feature minimalism. I think you have admitted yourself that “the way it used to be” is an impression from before a lot the HTML5 features built into the rendering engine and Firefox 4′s bad GC issues eventuated. Hence those sticking to 3.6 so long. MemShrink seems well on the way to fixing this bloat and Snappy seems to be focused on (finally?) moving lots of routines off the main thread and/or making Firefox (finally?) take advantage of multicore processors. Surely the bloat people speak of is more performance feel that is being addressed by MemShrink and Snappy rather than functionality or them overload?

            Perhaps a key illustration is the question of how Chrome’s actual chrome is coded. Is it JavaScript based and XUL-like as Firefox’s is? Or is it not only minimal looking but actually minimal in performance overhead compared to a JS+XUL driven chrome application like Firefox?

            Thanks for you consideration. I understand if you don’t have time to reply but please don’t delete me or chastise me for writing at some length. TL;DR is a depressing reflection on humanity to me.

        • Compartment per global recently landed, which would explain the huge increase in the number of user/system compartments.

          • Hi Wes

            I’ve read about CPG but don’t understand it all that deeply. Does it related to the Ghost Windows section under about:compartments at all?

            Anyway, see my comments below re Wallflower and my modem config pages. Since I disabled Wallflower (even the 1.1 version), User Compartments have returned to normal. I still have unexplained high about:blank entries but otherwize no more Ghosted Windows and excessive User Compartments. Still a substantial list of System Compartments but they seem legitimate according to Nick.

  12. This is good news. I like that they are trying to reduce paging, but actually FF’s disk activity is not caused so much by the paging of addon memory, but by the caching of pages on the disk.

    The article also says that this helps machines with low memory. It maybe so, but on my 6GB RAM desktop, what I always do after I instal Firefox or when I create a new profile is to completely disable the disk cache and increase the memory cache to at least 1GB. This reduces the disk activity from FF to almost none, and web pages load a whole lot faster.

    Of course, disk thrashing is probably not such a big issue with silent SSDs, and since most laptops and tablets today come with SSDs , but with only 1 or 2 GB of RAM, I can understand why it’s more important to reduce the memory footprint than to reduce disk thrashing.

    If I had any say, I would ask that the FF developers consider enabling the memory-only caching of pages by default in FF. Aside from the speed improvement and reducing disk thrashing, it’s easy to see that even a memory cache as small as 400-500MB would be sufficient for caching approximately 20-30 web pages at the same time, but the average user probably won’t need to have that many pages open at the same time anyway.

    For machines with 500MB or less RAM, it makes more sense to use the disk cache, but maybe the RAM size could be detected at install time or at every FF startup, and the appropriate cache could be used based on that.

    • Hi Gabi,

      I think the disk cache is an awesome idea, mostly because of people with limited bandwidth or Data usage (such as mobile). And for speed, disk I/O is almost always faster then network I/O especially over the internet.

      I relied heavily on disk cache when stuck on a GPRS connection in the country, for example.

  13. Can you test with Roboform? Roboform seems to have major memory leaks in Firefox (but not other browsers) but is so useful that I can’t work without it. I’ve reported the issue to Siber systems but they say they can’t reproduce it.

  14. Nick, I’ve posted to hopefully fill in some blanks I missed in my original post;

    http://blog.kylehuey.com/post/21892343371/fixing-the-memory-leak#comment-524693082

  15. Just a quickie: is it possible that a website with a frame reloading every second for hours could cause memory to balloon beyond physical RAM limits? My modem’s status page does this.

  16. What does the number at the end of a User Compartment listing represent?

    Here’s a big number from my modem’s config web pages:

    http://192.168.1.1/doc/adsl.htm [1139]

    it’s now

    http://192.168.1.1/doc/adsl.htm [1242]

    just in the time it’s taken to write this post.

    • Nicholas Nethercote

      Hover over the number with the mouse cursor and you’ll get a tool-tip that says “This value is the sum of N memory reporters that have the same path”. In other words, you’ve got 1242 compartments for http://192.168.1.1/doc/adsl.htm. I’m not certain, but I’d guess the page is doing something silly like creating a new iframe every time it updates?

      • Wallflower 1.1 is still broken.

        I have good bad news :) After testing with all but two add-ons (Wallflower 1.1 and Memchaser) disabled in my long-running profile and then with a completely new profile, both times in Nightly, with only two tabs open (my modem’s config pages, about:compartments) I found that Wallflower causes memory usage to grow beyond physical memory limits.

        The great news is the various MemShrink developments gave me the tools to diagnose this problem quicksmart. about:compartments, about:memory, Memchaser all helped. Also I could kinda see a lot of MemShrink tweaks trying valiantly to solve the problem.

        I guess it’s possible that Memchaser and Wallflower conflict. I’ll test that if you think it will help.

        The modem’s config pages are full of framesets but strictly speaking not iframes. I can break it down in more detail tonight after work if it might help? A meta refresh reloads the ADSL status page every 2 seconds which. Crude but understandable. The source code does write the 4 column x 10 row status table using document.write as well. However a 4×10 column table isn’t that much data. I’m loathe to blame the modem’s software since websockets, webRTC or AJAX were not likely available at the time.

        Anyway, it’s Wallflower 1.1 that can’t handle a page reloading itself every 2 seconds. After about 5 hours, it will balloon the available physical memory and render the OS into a semi-conscious state.

        In attempting the naked-profile test, I had to install Wallflower 1.1 again and it had the yellow stripe “preliminarily reviewed’ status. Isn’t this the closest AMO comes to ‘this add-on is dodgy’ ? I reckon there needs to be a ‘this add-on can, sometimes, cause performance degradation’ status.

        Anyway, that’s the verdict. Whether version 1.1 that was recompiled with the latest Add-On SDK or not, Wallflower is a performance hog.

        Hope this helps. I’d throw a lot of details into a bugzilla bug but they still have me banned from that. I reckon I’ve served my time. Maybe Nick you could ask them to re-instate my account?

        • Nicholas Nethercote

          Are you certain that it’s Wallflower at fault? What happens if you leave the modem page open without any add-ons installed?

          I ask because https://bugzilla.mozilla.org/show_bug.cgi?id=750048 sounds similar, and that’s pretty clearly the page’s fault. It sounds like it’s the modem page’s fault. With compartment-per-global, every new compartment represents a window (a.k.a. global) object. If lots of compartments are being created it must be because the page is creating lots of window objects. That’s my understanding at least…

          • I’ll check Nick, when I get home this evening. Thanks.

          • Nick based on the testing I’ve done, it’s definitely Wallflower 1.1

            It could not be clearer in my mind. I didn’t even have Memchaser installed this time. It seems to create one User Compartment every time it reloads which is every 2 seconds.

            Let me know if you want any more details.

          • I did another double/triple check of this overnight. Running latest Nightly, for over 6 hours, with just my modem’s meta refresh every 2 seconds page running, and zero add-ons installed (excepting PDF.js which was installed by default and disabled), I had no problems with memory. It never topped 150MB and dropped back down to 100MB. All very reasonable.

            Would it be reasonable to conclude that there may well be a) some code in Wallflower 1.1 that is still leaking after being compiled with SDK 1.6.6; or b) Jetpack unfortunately has some undiscovered leaks still in it?

            I’m happy to do more testing if it will help.

          • Nicholas Nethercote

            Thanks for the update. At this point it would really help if we could reproduce the problem. Are you able to extract the code for the modem’s page in a manner that would allow someone else to run it?

          • Hi Nick. Hmm, I can view source and replicate lot from there as AFAIK most of the code is client-side but of course to get device data, there must be some server-side I presume. I’ll replicate as much from the server side as possible, maybe even find the firmware and hack into that but where/how would I go about presenting this to someone for their own use? What would be the most useful format/location?

            I imagine it might be difficult to replicate the entire setup, if that’s necessary, without the specific device. I guess it depends how deep you want to go. I’d be happy to send the device to you (we both live in Melbourne) if I can get my hands on another modem which a friend might be able to help me with. I wonder if it’s possible that merely replicating the same front-end code without any server-side would produce the same problem? Basically it’s a fairly simple frameset that meta refreshes every 2 seconds, writing out about 20 lines of document.writes every time.

          • Nicholas Nethercote

            If you just extract the HTML and client-side scripts, can you replicate the problem?

          • Nicholas Nethercote

            Kyle said just right-clicking and choosing “save page as…” might be good enough.

          • Interestingly the behavior doesn’t present itself after ripping the view source, partially reconstructing the code and running it from local disk. This is with the latest Nightly, a fresh profile and then installing Wallflower 1.1 but not restarting since Wallflower 1.1 is a restartless add-on. I did however see this in the error console:

            Timestamp: 15-May-2012 7:57:58 PM
            Warning: An unbalanced tree was written using document.write() causing data from the network to be reparsed. For more information https://developer.mozilla.org/en/Optimizing_Your_Pages_for_Speculative_Parsing

            Perhaps that’s a hint at a solution? Perhaps not. The line reference with that error (line 81) corresponds literally to the end of a script block that contains the 20 lines of document.write status code for the modem. The reference to ‘network’ in that error message seems odd given the page is running off local hard drive. I guess perhaps that’s just a generic message though.

            I’m happy to send the source to whomever, however. What is the preferred or a suitable medium? Email, Pastebin, JSFiddle … something else? I can clean it up as well since it’s quite old and therefore has nastiness such as font tags!

            One interesting point is that the code is clearly not written for HTML5 but uses the (for want of a better term) HTML5 doctype of just <html>. I wonder if this code is so old, the decision to abandon meaningful doctype declarations for HTML5 means that this code is being interpreted as HTML5 and the HTML5 parser is choking on it? I guess that’s a simplistic instinctive guess though.

          • Nicholas Nethercote

            So your extracted version doesn’t show the problem? Bummer. Still, if you could email it to me (nnethercote is the user name, mozilla.com is the domain) that’d be helpful. Thanks.

  17. [pain mode=frustration]
    s/a big problem/a feature/ (cit.)

    Memory consuption. Tsch. Last (almost) good firefox release was 4.o. I use it plain without extensions and guess what, 10 tabs are too memory/cpu expensive, for my 3 years old MacBook. But Hey, we have the 3D DOM inspector. And 3-months release just beacuse of Chrome Crazyness. Yeah. Great.
    [/pain]

    Sorry, but i cannot feel relieved enough.

    • Nicholas Nethercote

      I don’t understand your comment. But Firefox 4 was terrible in terms of memory consumption. Recent Firefox releases are *much* better.

  18. The recent nightly consumes dramatically memory and than crashes after 10s when viewing the push-logs. This is what I see after the crash is send:

    Firefox 15.0a1 Crash Report [@ EMPTY: no crashing thread identified; corrupt dump ]

    Here is the picture of a xperf trace to show the high CPU usage and memory allocation:

    http://dl.dropbox.com/u/5749744/Firefox/Firefox_nightly_CPU_mem_.png

  19. Hi. I think I’ve got the about:blank compartments identified. They seem to represent uncached tabs (glorified bookmarks) in the ‘foreground’ and ‘background’ (Panorama) tab groups.