Gecko
Tim Taubert set the style of a not-yet-restored tab’s <browser> element to display:none, eliminating the need for any layout data to be stored in memory for such a tab. This was a MemShrink:P1 bug and is a nice improvement for users who typically have many (e.g. 100+) tabs open, because it saves 0.17MB per not-yet-restored tab on 64-bit platforms, and slightly less on 32-bit platforms.
Bobby Holley fixed a defect that was causing some add-ons (NoScript, Roboform, and Google PageSpeed) to create ghost windows. This fix has been also been ported to the Aurora and Beta channels and so will be present in Firefox 16.
Add-on SDK
Version 1.10 of the Add-on SDK was released. It has two fixes that prevent various leaks. Furthermore, Alexandre Poirot fixed two other leaks in the Add-on SDK; these changes will presumably make it into version 1.11.
The Add-on SDK has had a lot of leaks fixed over the past few months. Hopefully we’re getting near the end of them!
Generational GC
I have started helping with the ongoing project to convert the JavaScript engine’s mark-and-sweep garbage collector to a compacting, generational collector. This is the #2 item on the current MemShrink “Big Ticket Items” list because it should significantly reduce JS memory consumption, and it should also help performance in other ways.
It’s a big task because it makes a very fundamental change to the JS engine: garbage-collected things such as JS objects, strings and functions can move. This is a big deal because these things are represented within the JS engine as C++ objects of various kinds, and it’s not normal for C++ objects to move! Think about it — if you have a pointer to a C++ object, and the object moves, the pointer is no longer valid.
I won’t go into detail about how we handle this but the important thing to know is that pretty much every place in the JS engine where we have a raw C++ pointer to a GC thing object needs to be manually changed to use a new representation. This is literally tens of thousands of changes that must be completed before the garbage collector can be made either compacting or generational (the two properties are orthogonal, with both providing benefits).
If you are interested in following the progress of this conversion, take a look at bug 753203.
A reminder
In these MemShrink reports I usually write about Firefox changes just after they first land on mozilla-central. Such changes will show up in Nightly builds within a day, but won’t reach an official Firefox release for 12–18 weeks, as per Firefox’s release schedule. If you remember that you’ll avoid any confusion about which changes are in which release.
For example: except where noted otherwise, the changes I’ve mentioned above will be in Firefox 18 — assuming they don’t cause problems that cause them to be backed out — which is due for final release in early January, 2013. (The current scheduled release date is January 1st, but that will probably be delayed slightly because New Year’s Day is not a good day for a release!)
Bug Counts
Here are the current bug counts.
- P1: 21 (-1/+0)
- P2: 89 (-5/+4)
- P3: 100 (-3/+7)
- Unprioritized: 3 (-2/+3)
21 replies on “MemShrink progress, week 65–66”
Hi Nicholas,
you asked me for the extensions I use [1]. I’ve pasted the relevant section from about:support to pastebin [2]. If you see anything out of order, please let me know. If not, it doesn’t really matter. I have to restart the browser daily anyway so the zombie compartments don’t even get the chance to accumulate over time. It would just be interesting to let the extension authors know so they get a chance to fix it.
Thanks again for the great work to you and the rest of the MemShrink team!
[1] https://blog.mozilla.org/nnethercote/2012/09/05/memshrink-progress-week-63-64/comment-page-1/#comment-7574
[2] http://pastebin.com/5LJNxcKh
If you want to try disabling some, based on little more than gut feeling I’d suggest the following: Enter Selects, Greasemonkey, Suspend Background tabs, Test Pilot, Textarea Cache.
Quick progress update:
I’ve decided to identify the leaking extensions the hard way. I switched off all extensions and reactivate a single one each day and check if it creates zombies/ghosts. This approach doesn’t cover all cases where extensions affect each other negatively but with 19 extensions this would be a combinatorial nightmare to test.
Day 4 results:
Adblock Plus: Not leaking
Element Hiding Helper: Not leaking
Enter Selects: Leaking. Ghost windows and zombies started appearing shortly after I turned it on again.
GreaseMonkey: Apparently not leaking but test still ongoing.
Enter Selects is a really tragic for me. Firefox has after 4 years still the best algorithm for matching history and bookmarks items on the market and Enter Selects puts this power at my fingertips. Unfortunately the frontend team decided to ignore this power when coming up with their own auto select (i.e. autofill) implementation and somehow managed to make it even worse than Chrome’s offering. 🙁
Is it possible that the problems with Enter Selects are caused by using an outdated version of the Addon SDK?
Thanks for investigating! It’s definitely possible that Enter Selects’ problem is caused by it using an old Add-on SDK. It hasn’t been updated since March this year, and numerous SDK leak fixes have occurred in that time.
What are the steps to reproduce the leaks?
FWIW, FF15 fixed some zombie compartments with Enter Selects (https://bugzilla.mozilla.org/show_bug.cgi?id=745149) but I guess there are still more problems.
It looks like there’s an effort underway to update all extensions hosted on AMO to the lastest Addon-SDK version [1]. Maybe this will also solve the problems with Enter Selects. I’ll definitely do another test once those updates are available.
I don’t have any concrete steps to reproduce the problems other than to simply use the problematic extension for some extended period. Right now I’m just trying to identify the subset of leaking extensions. Once this is finished I’ll look into creating minimal test cases or at least easily reproducible instructions. Hopefully this will be done by the time of your next MemShrink update.
[1] https://blog.mozilla.org/addons/2012/09/21/sdk-re-pack-progress/
All that memory leaking nonsense has been fixed. Finally.
For me the memory situation has been markedly improved since Firefox 15. That certainly deserves a ‘well done’! I was used to having to restart the browser every other day because of memory usage and CPU usage steadily increasing over time.
So now the memory no longer climbs noticeably. Surprisingly the CPU usage still ‘leaked’. On a hunch I tried disabling the BetterPrivacy extension I was using (I use about a dozen extensions), and that seems to have solved the CPU leak problem to a large extent for me. Though I realise this may be more of a Snappy bug, clearly having BetterPrivacy disabled reduces memory usage overall.
Can you report the slowness to the Better Privacy authors at http://nc.ddns.us/extensions.html? Thanks!
Done.
The author is clearly not interested in discussing the subject.
I found one thread in his support forum which might be related, in which the developer simply denies there is a problem.
Not much I can do to help, sorry 🙁
Bastiaan: Try taking it to Taras’ snappy blog, they’ve resolved similar addon issues before (eg: https://blog.mozilla.org/tglek/2012/07/26/snappy-july-26-go-try-the-gecko-profiler/, specifically bug 777266) through use of the profiler in the nightly builds of Fx.
I’ve noticed a potential leak/ghost windows issue in the latest nightly series.
When using a blank profile and visiting moosejaw.com I am left with ghost windows (from about:compartments) after the tabs have been closed, reduce memory pressed a number of times and the browser left idle for ~15 mins. I’m using nightly x86_64 Linux, I can’t reproduce it in the current stable, beta or aurora builds. I haven’t been able to find any similar bug reports on bugzilla.
I’m happy to submit it on bugzilla with further details, I just wanted to check if there are any other reports of this issue.
Thanks for all of your awesome work! 🙂
Thanks for the report! I submitted it and CC’d you: https://bugzilla.mozilla.org/show_bug.cgi?id=795221 🙂
Is there a place to report leaky add-ons other than bugzilla? I’ve noticed Pencil 1.3.4 leaks 100s of MB on my system after I close it (FF 15 and 16 beta). I tried to report it to the plug in author but I haven’t heard back.
In general, each add-on has its own bug reporting mechanism. If the author isn’t responding it may be because they aren’t actively maintaining the add-on any more.
But you’ve got my curiosity: what’s the nature of the leak? Do you have concrete steps to reproduce? Does about:memory tell you anything notable?
Memory reported below is in Private Working Set. Firefox 16 Beta (15 is similar) Win7 x64.
Steps to reproduce:
1. Start up firefox. It baselines at about 112MB
2. Start up Pencil and load one of my saved files. Memory shoots up to 331MB but then drops to 280 MB.
3. Close pencil.
4. Perform GC/CC/Minimize in about:memory. The number never goes down.
When I repeat the procedure I’ll see usage jump up abother 100MB and never be reclaimed. I’ll post about:memory shortly.
Pencil doesn’t work with Firefox 18, which is the current in-development version…
about:memory verbose: http://pastebin.com/tWJvLsay
As I was doing this I thought that it would be cool if one could go to about:memory and capture the usage at two discrete times and compare them with diff or something (pretty graphs?), similar to certain leak detectors. I know that it isn’t quite the point of about:memory but it would be useful for someone like me to look and say ‘oh hey the memory is being allocated here…’.
Yeah, a tree diff would be useful.