MemShrink progress, week 40

The past week was fairly quiet for MemShrink.

Add-ons

Our testing of the top 100 installed add-ons is progressing slowly.  We need more help here.  Please join in if you are interested!

The Image Zoom and Baow add-ons were downgraded to “preliminarily reviewed” status because they cause zombie compartments, as per the new AMO policy.  The main effect of the downgrade is that they’ll show up lower in AMO search results.

Miscellaneous

I modified the storage of CSS properties and values to minimize waste caused by alignment.  On 64-bit platforms this reduced the size of a property/value pair from 24 bytes to 18 bytes, which saves about 0.5MB for an instance of Gmail.  (Gmail uses a lots of CSS!)  On 32-bit platforms the reduction is 1/3 the size, with a property/value pair dropping from 12 to 10 bytes.

Justin Lebar capped the amount of memory used for decoded images.  Decoded images in background tabs are currently discarded 20-40 seconds after they stop being visible in the foreground tab.  With Justin’s patch, there is now a second criterion:  if the decoded image cap is exceeded, decoded images in background tabs will be discarded until the cap is met.  (Decoded images in the foreground tab are never discarded, so it’s possible that all decoded images in background tabs are discarded and the cap is still exceeded.)  The cap defaults to 50MiB, but it can be changed in about:config with the image.mem.max_decoded_image_kb option.  The main effect of this change is that decoded images in background tabs will be discarded more quickly than they used to be in some circumstances.

Bug counts

This week’s bug counts:

  • P1: 22 (-2/+0)
  • P2: 129 (-6/+3)
  • P3: 92 (-3/+3)
  • Unprioritized: 0 (-0/+0)

That’s a net reduction of five bugs!  Furthermore, we only had five bugs to triage in today’s MemShrink meeting.  Good signs.

23 Responses to MemShrink progress, week 40

  1. I hope you cache the dimensions so you a repaint of the page is not necessary if a tab is opened again.

    • My previous comment was very unclear :-)

      I meant to mention:

      I hope the dimentions of the images that get discarded are kept, so that repainting of the page isn’t necessary is a tab is opened again, even if some images need to be decoded.

  2. Hi Nicholas,

    My standard test using 12 beta 1.

    I opened about 60 tabs from my bookmarks and left the browser alone for about 20 minutes.

    680,000 kb.

    I then right clicked the Gmail tab and hit “Close Other Tabs”.

    The memory never dropped below 310,000 kb with just Gmail open.

    Now I am writing this comment and I have this web page open alongside http://mnmlist.com/ which is one of the most basic websites you can get.

    The memory usage still hasn’t dropped below 250,000 kb.

    I did the same test with Firefox 11, the release version and with just Gmail open, the memory managed to get to 240,000 kb and with just http://mnmlist.com/ open it then went down to 217,000.

    11 is better than 12 but 10.0.2 seemed to be better than both of these on the same system with no big changes in hardware or software using the same bookmarks.

  3. Hi,

    Thanks for the effort in reducing the memory usage. The memory usage for Firefox 11 on my home and work machines is much improved. When closing tabs, most memory appears to be released.

    I have noticed that one of my development web pages is leaving a zombie compartment, i.e. start firefox, load page in tab, close tab, about:memory -> minimize … compartment for page persists. Interestingly, the page and related objects contains no explicit javascript. I only use 2 extensions, and the zombie occurs with the extensions disabled. Unfortunately the page isn’t publicly available.

    The web page has a very simple structure. I.e. the body has 2 divs, each div contains 1 object, objects are a xml file with xslt transform to render the data in the xml file as xhtml containing a table [xslt done using xml-stylesheet directive in xml file]. The xml files if loaded seperately, do not leave zombie compartments.

    The dom section of the about:memory report contains window entries for the xml objects but there are no zombied compartments for the objects. I.e. the zombie is the page that includes the objects.

    It looks like the use of objects in the page may be causing the zombie compartment [or xslt transform of the xml].

    • Nicholas Nethercote

      It would be very useful to have a test case that demonstrates the problem. Are you able to modify or simplify the page somehow so that you would be able to make it publically available?

  4. “Preliminary review” doesn’t say exactly what it should for a downgraded module. I take it to mean the addon is very new and no one has gotten around to doing a full check; which doesn’t seem so bad because the maintainer of a freshly-written addon is often prompt to fix it. In the case of downgrades, that gives completely the wrong impression. May I suggest something like: “reviewed, needs further work” or “experimental (reexaminated 2012-02-14 / n weeks ago)”? I know this is an exceptional case and I don’t want the reviewers to come under pressure, but there’s probably a better wording. Or just switch to more specific warnings, like: “may cause increased memory use”.

  5. I’m sorry to bring this up again here in your blog, but this memory leak is significantly impacting Firefox’s usefulness to me.

    While logged in to Hotmail, memory usage climbs pretty quickly and I find I need to restart Firefox daily.

    Lots of tabs open – mem usage starts just over a Gig in the morning. If I never log in to my email, memory usage is close to the same at the end of the day.
    If I am logged in to Hotmail all day, memory usage climbs to well over 2 Gig and the browser gets very slow.

    I just wish a Firefox developer who knows the memory stuff could take a look at this. It’s pretty nasty and affects a lot of people. Maybe their web site is doing something horrible. I’d be happy if MS fixed it, too!

    - Aaron

    • Nicholas Nethercote

      Your “affects a lot of people” may not be true. I don’t recall hearing about problems with Hotmail, and if this was a common problem I would expect to have.

      So, the question is whether your Firefox installation is unusual in any way. The most likely cause is add-ons — do you have any installed? If so, are you able to disable them and see if you can still reproduce the problem? That would indicate if an add-on is at fault, which is very common; that would be very helpful information if you can provide it. Thanks!

  6. I will indeed do even more testing with add-ons disabled, but I did test that in older versions of FF and had the same big memory issue. That was before I’d isolated it to Hotmail.

    As to not hearing about it, I’ll reference this past comment in your own Blog:
    http://blog.mozilla.org/nnethercote/2011/11/09/memshrink-progress-week-21/#comment-4192

    And this post in FF support (errolt (same person) did test with add-ons disabled):
    https://support.mozilla.org/en-US/questions/892214

    Thanks for the response,
    - Aaron

  7. Aaron,

    Maybe time to file a bug and reference the FF support thread?

    Manoj

  8. I found a good already-filed bug on this problem:
    https://bugzilla.mozilla.org/show_bug.cgi?id=699489

    Nicholas already commented on it there. When I have time, I will do some proper testing with add-ons disabled. For a good test I need to have FF running for a long time so it will take me some days to get valid results. I’ll then add comments and supporting documents (like about:memory dumps) to that existing bug report.

    A while back I blamed AdBlock Plus for the memory problem, but I uninstalled it for a while and still had the problem.

    - Aaron

  9. FF11 is way better than FF10, nice! I no longer have to shut down FF due to >1gb memory usage plus another gb in swap.

  10. Talk, talk, talk … endless talk.

    I’ using Firefox 11 and I don’t see much improvement in the memory and CPU usage in comparison with the older versions. At the moment I have only two tabs open – this page and about:memory. Firefox is sluggish – sometimes I see a black window and sometimes during scrolling/writing it freezes for a brief moment. The memory usage is as follows:

    0.27 MB — canvas-2d-pixel-bytes
    1.13 MB — gfx-d2d-surfacecache
    19.37 MB — gfx-d2d-surfacevram
    2.13 MB — gfx-surface-image
    0.00 MB — gfx-surface-win32
    189.52 MB — heap-allocated
    374.32 MB — heap-committed
    49.36% — heap-committed-fragmentation
    2.55 MB — heap-dirty
    534.48 MB — heap-unallocated
    2 — js-compartments-system
    13 — js-compartments-user
    165.00 MB — js-gc-heap
    13.34 MB — js-gc-heap-arena-unused
    0.00 MB — js-gc-heap-chunk-clean-unused
    2.13 MB — js-gc-heap-chunk-dirty-unused
    88.82 MB — js-gc-heap-decommitted
    63.20% — js-gc-heap-unused-fraction
    2.38 MB — js-total-analysis-temporary
    3.53 MB — js-total-mjit
    41.42 MB — js-total-objects
    35.61 MB — js-total-scripts
    26.24 MB — js-total-shapes
    19.87 MB — js-total-strings
    1.29 MB — js-total-type-inference
    0 — low-memory-events-physical
    0 — low-memory-events-virtual
    637.05 MB — private
    593.84 MB — resident
    12.14 MB — shmem-allocated
    12.14 MB — shmem-mapped
    1,342.31 MB — vsize

    I visited a lot of sites prior to that. I have extensions – that’s one of the main reasons for which I use Firefox.

    I fail to see a long term solution to the memory/CPU usage problems.

    It is easy to blame the Firefox extensions since there is no practical way (for most users) to measure their real impact. Even if memory leak in some widely used extension is found and fixed this won’t guarantee that the next version of this extension won’t contain a memory leak. That also applies to Firefox itself – with every new version more features are added, the complexity increases and that creates more opportunities for new memory/CPU problems.

    It is impractical to test every possible usage scenario of Firefox and its extensions in order to reduce the memory/CPU problems. It is unrealistic to expect that the Firefox users will do that for free – some technophiles will do that, but most users will not, even if they can, because the time and efforts needed to do that properly are not free.

  11. Is there any place to find changelogs for the different versions? I saw a horrendous font rendering issue in 12b1 that went away in 12b2, though both versions had crashes on certain pages. I’m just curious to know which bug it was I was seeing the first time; I figure the crash issue will be resolved on its own. I’ve tried looking up changelogs before but they’re Google-proof if they exist. Worse, Google isn’t smart about version numbers so looking up firefox 12 often pulls up 4.0b12 or useless junk like that. Changelogs used to be published with every release, but I have no idea where to find them now.

    I know this isn’t your department, but the whole team would probably do well to know that Bugzilla is painfully difficult for inexperienced users to navigate, and I’ve never had good success searching there–even for issues I already knew existed. There’s really no good resource I’ve found to explain how the whole process works, either.

  12. Nicholas, I think LastPass extension is still leaking to heap-unclassified and it’s somehow related to popups and other installed extensions. I tested like this:

    1. Install latest Nightlly build. I used 2012-03-26 32-bit build on Windows 7 Professional 64-bit.
    2. Create new profile.
    3. Install “LastPass 1.90.06″ (https://addons.mozilla.org/en-US/firefox/addon/lastpass-password-manager).
    4. Install “Restartless Restart 8″ (https://addons.mozilla.org/en-US/firefox/addon/restartless-restart).
    5. Restart Firefox.
    6. Login to LastPass account using extension.
    7. Open a new tab with about:memory. Heap-unclasiffied is ~ 7-8Mb.
    8. Open a new tab with http://www.gsmarena.com/apple_ipad_3-review-739.php
    9. Click on the first iPad image in the review – a popup opens.
    10. Close the popup.
    11. Repeat 9-10 steps 20 times.
    12. Go to about:memory tab and click “Minimize memory usage” button.
    13. Check line with heap-unclassified – it is ~75Mb already and never goes down.

    The weirdest thing is that this doesn’t happen if only one of those two extensions is enabled – heap-unclassified grows only to ~17-18Mb.

    I think you can get similar result with LastPass and some other extensions too, e.g. TabGroups Menu.
    Maybe it’s Jetpack related?

  13. Nicholas, thank you for your continued work on the MemShrink project. I notice some definite improvements in Firefox 11 especially with the LastPass memory leaks fixed.

    I noticed that the page https://wiki.mozilla.org/Performance/MemShrink#Meetings has not been updated to point to your newest weekly meeting minutes. I found the link to weeks 38, 39, and 40 on the week 37 page. You might want to keep the main page updated to point to the minutes for each week so more people can find the weekly minutes.

    Thank you,
    Peter

  14. Thanks for the effort in reducing the memory usage. The memory usage for Firefox 11 on my home and work machines is much improved. When closing tabs, mo

  15. effort in reducing the memory usage. The memory usage for Firefox 11 on my home and work machines is much improved. When closing tabs, mo