A quieter week this week. Well, plenty of work was done but not yet completed, and I mostly write only about changes that have been finished.
- Brian Hackett finished up the last of his memory usage reductions for JavaScript type inference. The previous attempted merge of type inference bounced due to unacceptable memory usage, but Brian has reduced its usage massively and it will land soon.
- Dave Hunt finished the MemBuster-like endurance test. This will help us catch regressions in the future. (Update: Dave has described how to run the tests yourself.)
- Sander van Veen added a facility to identify system compartments created to hold sandboxes, which means they can be identified in about:memory. If you have written an add-on that creates sandboxes, please consider modifying your calls to
Components.utils.Sandboxto specify asandboxNameproperty in the optional parameter. See the documentation for details. - Update 2: I forgot to mention that Wan-Teh Chang landed my fix to prevent waste due to heap allocation round-ups in NSPR’s PLArena and backported it to mozilla-central. This allowed me to remove the nsPresArena workaround that I added last week (which in extreme cases could save 100s of MB). It also avoids a megabyte or two of wasted here and there in other places where PLArena is used.
Now for the bug counts. (Canned MemShrink bug searches are available here.)
- P1: 29 (-5, +4)
- P2: 66 (-6, +8)
- P3: 37 (-1, +2)
- Unprioritized: 4 (-1, +4)
There was lots of P1 movement, which is good: a couple were fixed, some have had enough progress made on them that they were able to be downgraded to P2, and some new problems/opportunities were identified.
17 replies on “MemShrink progress, week 10”
When do we see the improvements? Firefox 9?
This is such a pleasure to read this stuff. Makes me wish there was a “developer love” tool where appreciative users could write thank you emails to developers and vote for the best superstar developer of the day/month/week/year. We need more love for the developers who make the big differences under the hood!
Nicholas is there a fixed time frame for the MemShrink initiative?
It’s just brilliant to hear that long-term regression management is being given attention. This cannot be prioritized too high IMHO. There’s an areweslimyet.com or something isn’t there? If not, there should be so ordinary non-developers can keep an eye on memory management performance. As some 3rd/4th-party politicians say here is Australia: Got to keep the bastards honest! Heh Heh.
W.r.t. time-frames, https://wiki.mozilla.org/Performance/MemShrink says: “Ideally, these weekly meetings would only be needed for a medium length of time, say 1 or 2 quarters. Hopefully after that, things will have improved enough that meetings could be less frequent (bi-weekly or monthly) or not needed at all.” Six months is looking more likely at the moment, we’ll see how things are going then.
Nicholas I am wondering if there are any way to tell which particular tabs / site is leaking memory? Everyday i have RSS Reading with up to 100s tabs opened. Sometime I know immediately that somewhere is leaking memory when memory Firefox is slow to response and Mem usage was higher then normal.
If you look at the per-compartment reports in about:memory you have some chance of working it out, if it’s a leak involving JS code (as many leaks do). Look for the URL of the biggest compartment, that may give you a clue. Note that plenty of sites run scripts from other sites (esp. from google.com, facebook.com and twitter.com), which can make things harder to follow.
NJN what do you think about a new view in about:memory that would map tabs or sites to compartments? This could help identify websites that use a lot of memory by calling upon a lot of third-party sites, a la the plethora of ‘share me’ buttons on the web. A tree view could be very useful I reckon. What do you think? Would it be a lot of work?
If it was for all memory, not JS, I think it would be more compelling. It sounds like this is difficult, though (see http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/0a5c90a233bc4871/b3107a003d96d88f?#b3107a003d96d88f).
Would in-memory compression help reducing memory budget for ressources not currently displayed in active tab but nonetheless cached (typically other undisplayed tabs and windows) ?
Rgds
Nope, Javascript must still run, so you can’t compress anything.
We do discard decompressed images in background tabs after a short time (10–20 seconds, if I remember correctly).
Awesome works!! Keep up Nicolas.. You really changed my experience with Firefox. I tried latest Nightly on my old PC (some specs: 1.7 GHz Pentium 4, 512 MB DDR RAM) and Nightly was consuming just 91 MB with two tabs which previously use to hike 110 MB on one tab. That`s some real efforts by you and your subordinates..
Salute you!!!
I was testing bug 643651, my Nightly is updated to yesterday of current date (current one not released), I am no longer seeing huge memory leak in Test URL. You should test it again.
@njn
https://bugzilla.mozilla.org/show_bug.cgi?id=230675#c12
Comment says it can save memory allocation as well so you should check how much memory it saves if worthy include it in MemShrink.
Nicholas, I´ve went to a trip, left my pc and firefox 8 alfa 2 running for 5 days, and when I’ve came back my cpu was hot like crazy with firefox using 100% of one of the cores (core i5 520m, 8gb RAM) and with a 1,6gb memory consumption, while when I restart with the same tabs it uses only 400mb. So I don’t know where the progress is going but certainly it’s not working because there’s a serious bug that is very present and which causes the memory consumption to raise like mad (mostly in the night, which is very strange, midnight dinner???) and after some time the cpu usage goes high also (probably because of the gigantic memory usage).
http://blog.mozilla.org/nnethercote/2011/08/29/browser-x-is-using-y-mb-of-memory-with-z-tabs-open-is-a-meaningless-observation/
What sites you had open matters.Web site itself may be responsible for leaking memory (and browser can’t help it in such a case). Second, a lot of add-on expose latent bugs in Firefox, causing it to leak memory, or just slowly leak memory themselves.
OK afeter updating to alfa2 08-28-11 its not having the memory midnight dinner anymore and is not raising memory usage just because of lefting firefox running for days. I believe there were improvements.
Nicholas: Thank you for your efforts in the recent MemShrink project.
It’s great to see mozilla start to work on performance improvement seriously, and the improvement is also quite pleasant. But it’s just a start, I’ve tried the 9.0 a1 nightly these days, and also check the MemShrink bug list, both usage experience and the bug lists tell me that there’re great many things should be done before you slow down the MemShrink and other performance concern projects'(startup speed, UI responsiveness, etc) efforts. I doubt whether 6 months would be enough.
Because the foremost important reason more and more users migrate to Chrome is just for performance(or more precisely user experience). My 2 cents is that even more than 50% of mozilla developers and the contributors are dedicated to improve performance is not a overkill at the moment. You might have the same experience about the users complaint. Firefox is most feature rich, especially when it cooperate with vastly amount add-ons available, just the performance destroy it, and drive the users away. Slow down the new feature implementation and put more efforts in improving performance is quite acceptable to me.
Btw, I’ve read that there’s a project to make add-ons default to compatible in the 9.0. Hurry and this is critical to firefox.