Categories
add-ons Firefox Memory consumption MemShrink

MemShrink progress, week 40

The past week was fairly quiet for MemShrink.

Add-ons

Our testing of the top 100 installed add-ons is progressing slowly.  We need more help here.  Please join in if you are interested!

The Image Zoom and Baow add-ons were downgraded to “preliminarily reviewed” status because they cause zombie compartments, as per the new AMO policy.  The main effect of the downgrade is that they’ll show up lower in AMO search results.

Miscellaneous

I modified the storage of CSS properties and values to minimize waste caused by alignment.  On 64-bit platforms this reduced the size of a property/value pair from 24 bytes to 18 bytes, which saves about 0.5MB for an instance of Gmail.  (Gmail uses a lots of CSS!)  On 32-bit platforms the reduction is 1/3 the size, with a property/value pair dropping from 12 to 10 bytes.

Justin Lebar capped the amount of memory used for decoded images.  Decoded images in background tabs are currently discarded 20-40 seconds after they stop being visible in the foreground tab.  With Justin’s patch, there is now a second criterion:  if the decoded image cap is exceeded, decoded images in background tabs will be discarded until the cap is met.  (Decoded images in the foreground tab are never discarded, so it’s possible that all decoded images in background tabs are discarded and the cap is still exceeded.)  The cap defaults to 50MiB, but it can be changed in about:config with the image.mem.max_decoded_image_kb option.  The main effect of this change is that decoded images in background tabs will be discarded more quickly than they used to be in some circumstances.

Bug counts

This week’s bug counts:

  • P1: 22 (-2/+0)
  • P2: 129 (-6/+3)
  • P3: 92 (-3/+3)
  • Unprioritized: 0 (-0/+0)

That’s a net reduction of five bugs!  Furthermore, we only had five bugs to triage in today’s MemShrink meeting.  Good signs.

23 replies on “MemShrink progress, week 40”

I hope you cache the dimensions so you a repaint of the page is not necessary if a tab is opened again.

My previous comment was very unclear 🙂

I meant to mention:

I hope the dimentions of the images that get discarded are kept, so that repainting of the page isn’t necessary is a tab is opened again, even if some images need to be decoded.

Hi Nicholas,

My standard test using 12 beta 1.

I opened about 60 tabs from my bookmarks and left the browser alone for about 20 minutes.

680,000 kb.

I then right clicked the Gmail tab and hit “Close Other Tabs”.

The memory never dropped below 310,000 kb with just Gmail open.

Now I am writing this comment and I have this web page open alongside http://mnmlist.com/ which is one of the most basic websites you can get.

The memory usage still hasn’t dropped below 250,000 kb.

I did the same test with Firefox 11, the release version and with just Gmail open, the memory managed to get to 240,000 kb and with just http://mnmlist.com/ open it then went down to 217,000.

11 is better than 12 but 10.0.2 seemed to be better than both of these on the same system with no big changes in hardware or software using the same bookmarks.

Hi,

Thanks for the effort in reducing the memory usage. The memory usage for Firefox 11 on my home and work machines is much improved. When closing tabs, most memory appears to be released.

I have noticed that one of my development web pages is leaving a zombie compartment, i.e. start firefox, load page in tab, close tab, about:memory -> minimize … compartment for page persists. Interestingly, the page and related objects contains no explicit javascript. I only use 2 extensions, and the zombie occurs with the extensions disabled. Unfortunately the page isn’t publicly available.

The web page has a very simple structure. I.e. the body has 2 divs, each div contains 1 object, objects are a xml file with xslt transform to render the data in the xml file as xhtml containing a table [xslt done using xml-stylesheet directive in xml file]. The xml files if loaded seperately, do not leave zombie compartments.

The dom section of the about:memory report contains window entries for the xml objects but there are no zombied compartments for the objects. I.e. the zombie is the page that includes the objects.

It looks like the use of objects in the page may be causing the zombie compartment [or xslt transform of the xml].

It would be very useful to have a test case that demonstrates the problem. Are you able to modify or simplify the page somehow so that you would be able to make it publically available?

“Preliminary review” doesn’t say exactly what it should for a downgraded module. I take it to mean the addon is very new and no one has gotten around to doing a full check; which doesn’t seem so bad because the maintainer of a freshly-written addon is often prompt to fix it. In the case of downgrades, that gives completely the wrong impression. May I suggest something like: “reviewed, needs further work” or “experimental (reexaminated 2012-02-14 / n weeks ago)”? I know this is an exceptional case and I don’t want the reviewers to come under pressure, but there’s probably a better wording. Or just switch to more specific warnings, like: “may cause increased memory use”.

I’m sorry to bring this up again here in your blog, but this memory leak is significantly impacting Firefox’s usefulness to me.

While logged in to Hotmail, memory usage climbs pretty quickly and I find I need to restart Firefox daily.

Lots of tabs open – mem usage starts just over a Gig in the morning. If I never log in to my email, memory usage is close to the same at the end of the day.
If I am logged in to Hotmail all day, memory usage climbs to well over 2 Gig and the browser gets very slow.

I just wish a Firefox developer who knows the memory stuff could take a look at this. It’s pretty nasty and affects a lot of people. Maybe their web site is doing something horrible. I’d be happy if MS fixed it, too!

– Aaron

Your “affects a lot of people” may not be true. I don’t recall hearing about problems with Hotmail, and if this was a common problem I would expect to have.

So, the question is whether your Firefox installation is unusual in any way. The most likely cause is add-ons — do you have any installed? If so, are you able to disable them and see if you can still reproduce the problem? That would indicate if an add-on is at fault, which is very common; that would be very helpful information if you can provide it. Thanks!

I will indeed do even more testing with add-ons disabled, but I did test that in older versions of FF and had the same big memory issue. That was before I’d isolated it to Hotmail.

As to not hearing about it, I’ll reference this past comment in your own Blog:
http://blog.mozilla.org/nnethercote/2011/11/09/memshrink-progress-week-21/#comment-4192

And this post in FF support (errolt (same person) did test with add-ons disabled):
https://support.mozilla.org/en-US/questions/892214

Thanks for the response,
– Aaron

I found a good already-filed bug on this problem:
https://bugzilla.mozilla.org/show_bug.cgi?id=699489

Nicholas already commented on it there. When I have time, I will do some proper testing with add-ons disabled. For a good test I need to have FF running for a long time so it will take me some days to get valid results. I’ll then add comments and supporting documents (like about:memory dumps) to that existing bug report.

A while back I blamed AdBlock Plus for the memory problem, but I uninstalled it for a while and still had the problem.

– Aaron

Talk, talk, talk … endless talk.

I’ using Firefox 11 and I don’t see much improvement in the memory and CPU usage in comparison with the older versions. At the moment I have only two tabs open – this page and about:memory. Firefox is sluggish – sometimes I see a black window and sometimes during scrolling/writing it freezes for a brief moment. The memory usage is as follows:

0.27 MB — canvas-2d-pixel-bytes
1.13 MB — gfx-d2d-surfacecache
19.37 MB — gfx-d2d-surfacevram
2.13 MB — gfx-surface-image
0.00 MB — gfx-surface-win32
189.52 MB — heap-allocated
374.32 MB — heap-committed
49.36% — heap-committed-fragmentation
2.55 MB — heap-dirty
534.48 MB — heap-unallocated
2 — js-compartments-system
13 — js-compartments-user
165.00 MB — js-gc-heap
13.34 MB — js-gc-heap-arena-unused
0.00 MB — js-gc-heap-chunk-clean-unused
2.13 MB — js-gc-heap-chunk-dirty-unused
88.82 MB — js-gc-heap-decommitted
63.20% — js-gc-heap-unused-fraction
2.38 MB — js-total-analysis-temporary
3.53 MB — js-total-mjit
41.42 MB — js-total-objects
35.61 MB — js-total-scripts
26.24 MB — js-total-shapes
19.87 MB — js-total-strings
1.29 MB — js-total-type-inference
0 — low-memory-events-physical
0 — low-memory-events-virtual
637.05 MB — private
593.84 MB — resident
12.14 MB — shmem-allocated
12.14 MB — shmem-mapped
1,342.31 MB — vsize

I visited a lot of sites prior to that. I have extensions – that’s one of the main reasons for which I use Firefox.

I fail to see a long term solution to the memory/CPU usage problems.

It is easy to blame the Firefox extensions since there is no practical way (for most users) to measure their real impact. Even if memory leak in some widely used extension is found and fixed this won’t guarantee that the next version of this extension won’t contain a memory leak. That also applies to Firefox itself – with every new version more features are added, the complexity increases and that creates more opportunities for new memory/CPU problems.

It is impractical to test every possible usage scenario of Firefox and its extensions in order to reduce the memory/CPU problems. It is unrealistic to expect that the Firefox users will do that for free – some technophiles will do that, but most users will not, even if they can, because the time and efforts needed to do that properly are not free.

Google pays for Chrome security bugs between $500 and $3133.70 – http://www.chromium.org/Home/chromium-security/vulnerability-rewards-program

Some of these bugs may only affect the bank account of the security researcher, but the vulnerability rewards program of Chrome is a clear public message, which Google exploits well – http://www.computerworld.com/s/article/9224701/Google_puts_1M_on_the_line_for_Chrome_exploit_rewards

Mozilla can offer performance reward program for Firefox, not only to improve the performance and reduce the memory problems (thanks to more professional bug reports) but also to attract more public attention and possibly more browser users.

Is there any place to find changelogs for the different versions? I saw a horrendous font rendering issue in 12b1 that went away in 12b2, though both versions had crashes on certain pages. I’m just curious to know which bug it was I was seeing the first time; I figure the crash issue will be resolved on its own. I’ve tried looking up changelogs before but they’re Google-proof if they exist. Worse, Google isn’t smart about version numbers so looking up firefox 12 often pulls up 4.0b12 or useless junk like that. Changelogs used to be published with every release, but I have no idea where to find them now.

I know this isn’t your department, but the whole team would probably do well to know that Bugzilla is painfully difficult for inexperienced users to navigate, and I’ve never had good success searching there–even for issues I already knew existed. There’s really no good resource I’ve found to explain how the whole process works, either.

The “target milestone” field in Bugzilla tells you which version a patch lands in (e.g. “mozilla12” is Firefox 12). I don’t know how to determine which Beta version a patch landed in, though.

Nicholas, I think LastPass extension is still leaking to heap-unclassified and it’s somehow related to popups and other installed extensions. I tested like this:

1. Install latest Nightlly build. I used 2012-03-26 32-bit build on Windows 7 Professional 64-bit.
2. Create new profile.
3. Install “LastPass 1.90.06” (https://addons.mozilla.org/en-US/firefox/addon/lastpass-password-manager).
4. Install “Restartless Restart 8” (https://addons.mozilla.org/en-US/firefox/addon/restartless-restart).
5. Restart Firefox.
6. Login to LastPass account using extension.
7. Open a new tab with about:memory. Heap-unclasiffied is ~ 7-8Mb.
8. Open a new tab with http://www.gsmarena.com/apple_ipad_3-review-739.php
9. Click on the first iPad image in the review – a popup opens.
10. Close the popup.
11. Repeat 9-10 steps 20 times.
12. Go to about:memory tab and click “Minimize memory usage” button.
13. Check line with heap-unclassified – it is ~75Mb already and never goes down.

The weirdest thing is that this doesn’t happen if only one of those two extensions is enabled – heap-unclassified grows only to ~17-18Mb.

I think you can get similar result with LastPass and some other extensions too, e.g. TabGroups Menu.
Maybe it’s Jetpack related?

Nicholas, thank you for your continued work on the MemShrink project. I notice some definite improvements in Firefox 11 especially with the LastPass memory leaks fixed.

I noticed that the page https://wiki.mozilla.org/Performance/MemShrink#Meetings has not been updated to point to your newest weekly meeting minutes. I found the link to weeks 38, 39, and 40 on the week 37 page. You might want to keep the main page updated to point to the minutes for each week so more people can find the weekly minutes.

Thank you,
Peter

Comments are closed.