Year: 2012

Categories
about:memory AdBlock Plus add-ons compartments Firefox Memory consumption MemShrink

MemShrink progress, week 30

Add-ons

This was the week of add-ons in MemShrink-land.

Jared Wein fixed a problem in Firefox that was causing zombie compartments if you viewed a native video with any add-on installed that implements the nsIContentPolicy interface.  Examples of such add-ons are Adblock Plus, GreaseMonkey, and NoScript, which are respectively the #1, #3 and #9 most popular add-ons on AMO!  Welcome to the MemShrink club, Jared.

Speaking of GreaseMonkey, Arantius fixed a bug in it that was causing zombie compartments on some GM scripts when opening a background tabs.

I can’t tell who was responsible for the next fix, because the Add-on SDK folks use Github in a way I don’t understand.   But Myk Melez and/or Gabor Krizsanits greatly reduced the number of compartments used in JetPack-style add-ons.  In one example, the number of compartments dropped from 156 to 8, saving about 20MB of memory.  This was a MemShrink:P1 bug.

Also, as far as I can tell, the same patch also fixed bug 680821 which means that all compartments that hold sandboxes created by JetPack-style add-ons will be marked as belonging to that add-on in about:memory.

Jordan Miner reported that the Delicious Bookmarks add-on is leaking excessive numbers of connections to the Places database.  I tried to reproduce the problem and failed, but I don’t have a Delicious account.  If anyone else who does have a Delicious account is able to reproduce, please let me know or comment in the bug.

Finally, Jorge Villalobos and Andrew Williamson from the add-ons team came to this week’s MemShrink meeting.  We had a very fruitful discussion about how to help add-on authors detect and avoid memory leaks, and how to help add-on users understand which add-ons have high memory consumption.  Stay tuned for more details in the coming weeks!

Three Cheers for New Contributors

Krzysztof Kotlenga, a new contributor, removed an OpenGL cache that was wasting 1GB+ of memory on Linux when hardware acceleration was enabled (it’s currently not enabled by default, but will be at some point in the future).   Great work, Krzysztof!  [Update: I originally wrote WebGL instead of OpenGL, which was incorrect.]

Bug Counts

Here are the current bug counts.

  • P1: 26 (-2/+0)
  • P2: 132 (-14/+3)
  • P3: 67 (-2/+5)
  • Unprioritized: 4 (-0/+4)

That’s a net reduction of six bugs.  The main factor here was that I went through some of our P2s and closed ones that were stale and/or covered by other bugs, and downgraded to P3 a few more that are now known to be less important than we first thought.

Before finishing, I’d like to highlight bug 703427.  Richard Hipp, one of the SQLite developers, has a tiny patch that drastically reduces the amount of memory used by SQLite in Firefox — I ran with it for a while and my SQLite memory consumption dropped from ~15MB to less than 3MB.  The patch also causes a moderate speed drop, but it’s unclear if that speed drop is noticeable to Firefox.  We need someone who understands Firefox’s SQLite usage well to evaluate this patch so that Richard knows if it’s something that should go into a released version of SQLite as an option.  The bug is assigned to Marco Bonardo but he’s currently very busy.  Is there anyway else who knows enough about SQLite to take on this bug?

Categories
Build system Firefox

A fix for an 11-year-old bug in the build system

I have noticed for months that when building Firefox, sometimes configure will run when it shouldn’t, e.g. when I haven’t changed any makefiles or other build system files.  This is annoying, because it increases the time taken for a complete no-change build on my Linux box from ~50 seconds to ~70 seconds.  And it happens reasonable frequently, but I haven’t been able to reproduce it reliably.

Until yesterday, that is! I realized it happened reliably if I alternated building two different objdirs (e.g. debug and opt).  Once I could reproduce it reliably, it wasn’t hard to use make’s -d option to find out what was causing configure to re-run.  The answer was a file called $(topsrcdir)/.mozconfig.mk.  Because it is put in $(topsrcdir) instead of $(objdir), the building of one objdir can affect the building of another objdir.

The fix was pretty easy — it’s hard to move .mozconfig.mk out of $(topsrcdir), but there’s no good reason to re-run configure every time it changes, because it doesn’t contain anything that affects configure. So it was a one-line change in clienk.mk.

Kyle Huey tells me this bug has been present since 2000!

Categories
add-ons Firefox MemShrink

MemShrink progress, weeks 28–29

It’s been a quiet couple of weeks, due to Christmas, New Year, and all that.

Two leaks in add-ons were fixed.

  • cyberscorpio fixed a zombie compartment in the Super Start add-on.  The problem was that the add-on was failing to remove an event observer when a page is unloaded.  The fix will be present in v3.6.2.
  • Jesse Hakanen fixed a zombie compartment in the AdBlock Plus Pop-up Addon add-on (note that this is add-on extends the blocking functionality of AdBlock Plus, it’s not AdBlock Plus itself).  The problem was that the add-on was failing to catch the page unload event, and so some references to closed pages were left in memory.  The fix is now available in v0.3.

Because both leaks had similar causes, I updated the documentation on avoiding zombie compartments in add-ons accordingly.

Patches for two other MemShrink bugs landed.

I saw two noteworthy mentions of Firefox’s recent reductions in memory consumption. The first was in webmonkey.com’s review of Firefox 9.

Alongside the faster JavaScript processing Firefox 9 continues to show improvements from Mozilla’s MemShrink project, an ongoing effort to reduce memory usage in the browser. Indeed, for the first time in a very long time my testing showed Firefox 9 using less memory than Opera (which has long been the least RAM-hungry browser I test). Opening the same dozen tabs in both Firefox and Opera used only 367MB of RAM in Firefox compared to 378MB in Opera 11.60 [Update: Note that the memory test was performed with the following Firefox add-ons running: AdBlock, Ghostery, BetterPrivacy and HTTPS-Everywhere.] There’s no longer much difference between the two, which is a testament to Firefox’s dramatic improvement over the last six months of MemShrink efforts.

Cross-browser memory comparisons are fraught with difficulties, both in the measuring and the interpreting, so I don’t give the comparison with Opera much weight.  But the fact that this author has seen Firefox’s own memory consumption drop over the last few versions is meaningful.

The second was a comment on the previous MemShrink progress report from AC about Firefox 10.

You may remember that I posted a few weeks ago saying that Firefox 9 beta was great and that I could definitely see more memory and performance improvements over Fire 7 and 8. Well I am pleased to say that after a few days of using Firefox 10 beta 1, I can clearly see more memory improvements again. There is an obvious difference between Firefox 9 and Firefox 10 on the beta channel.

I have no hard evidence as such other than saying that nothing has changed on my laptop in the last few weeks but I am using the same heavy tab load and the memory usage displayed by Windows task manager is lower than it was by about 20 MB and the responsiveness and speed feels sharp.

Whatever you guys are doing with the Memshrink project, keep doing it because the results are paying dividends.

I’ve read literally 1000s of complaints about Firefox’s memory consumption online, so it’s very encouraging to hear compliments.

Here are the current bug counts.

  • P1: 28 (-0/+2)
  • P2: 143 (-1/+5)
  • P3: 64 (-1/+1)
  • Unprioritized: 0 (-0/+0)

No big changes, unsurprisingly.  [Update: the P2 numbers were incorrect and have been fixed.]