Category Archives: Garbage Collection

Notes on Reducing Firefox’s Memory Consumption

I gave a talk yesterday at the linux.conf.au Browser MiniConf, held in Ballarat, Australia.  Its title was “Notes On Reducing Firefox’s Memory Consumption”.

Below are the slides and notes in a SlideShare embedding. If you find that embedding problematic (some people do) you may prefer to download the PDF version directly.

MemShrink progress, week 21

MemShrink:P1 Bugs fixed

Terrence Cole made a change that allows unused arenas in the JS heap to be decommitted, which means they more or less don’t cost anything.  This helps reduce the cost of JS heap fragmentation, which is a good short-term step while we are waiting for a compacting garbage collector to be implemented.  Terrence followed it up by making the JS garbage collector do more aggressive collections when many decommitted arenas are present.

Justin Lebar enabled jemalloc on MacOS 10.7.  This means that jemalloc is finally used on all supported versions of our major OSes: Windows, Mac, Linux and Android.  Having a common heap allocator across these platforms is great for consistency of testing and behaviour, and makes future improvements involving jemalloc easier.

Gabor Krizsanits created a new API in the add-on SDK that allows multiple sandboxes to be put into the same JS compartment.

Other Bugs Fixed

I registered jemalloc with SQLite’s pluggable allocator interface.  This had two benefits.  First, it means that SQLite no longer needs to store the size of each allocation next to the allocation itself, avoiding some clownshoes allocations that wasted space.  This reduces SQLite’s total memory usage by a few percent.  Second, it makes the SQLite numbers in about:memory 100% accurate;  previously SQLite was under-reporting its memory usage, sometimes significantly.

Relatedly, Marco Bonardo made three changes (here, here and here) that reduce the amount of memory used by the Places database.

Peter Van der Beken fixed a cycle collector leak.

I tweaked the JavaScript type inference memory reporters to provide better coverage.

Jiten increased the amount of stuff that is released on memory pressure events, which are triggered when Firefox on Android moves to the background.

Finally, I created a meta-bug for tracking add-ons that are known to have memory leaks.

Bug Counts

I accidentally deleted my record of the live bugs from last week, so I don’t have the +/- numbers for each priority this week.

  • P1: 29 (last week: 35)
  • P2: 126 (last week: 116)
  • P3: 59 (last week: 55)
  • Unprioritized: 0 (last week: 5)

The P1 result was great this week — six fewer than last week.  Three of those were fixed, and three of those I downgraded to P2 because they’d been partially  addressed.

For a longer view of things, here is a graph showing the MemShrink bug count since the project started in early June.

memshrink bug count

There was an early spike as many existing bugs were tagged with “MemShrink”, and a smaller spike in the middle when Marco Castellucio tagged a big pile of older bugs.  Other than that, the count has marched steadily upward at the rate of about six per week.  Many bugs are being fixed and definite improvements are being made, but this upward trend has been concerning me.

Future Directions

So in today’s MemShrink meeting we spent some time discussing future directions of MemShrink.  Should we continue as is?  Should we change our focus, e.g. by concentrating more on mobile, or setting some specific targets?

The discussion was long and wide-ranging and not easy to summarize.  One topic was “what is the purpose of MemShrink?”  The point being that memory usage is really a secondary measure.  By and large, people don’t really care how many MB of memory Firefox is using;  they care how responsive it is, and it’s just assumed that reducing memory usage will help with that.  With that in mind, I’ll attempt to paraphrase and extrapolate some goals (apologies if I’ve misrepresented people’s opinions).

  • On 64-bit desktop, the primary goal is that Firefox’s performance should not degrade after using it heavily (e.g. many tabs) for a long time.  This means it shouldn’t page excessively, and that operations like garbage collection and cycle collection shouldn’t get slower and slower.
  • On mobile, the primary goal probably is to reduce actual memory usage.  This is because usage on mobile tends to be lighter (e.g. not many tabs) so the longer term issues are less important.  However, Firefox will typically be killed by the OS if it takes up too much memory.
  • On 32-bit desktop, both goals are relevant.

As for how these goals would change our process, it’s not entirely clear.  For desktop, it would be great to have a benchmark that simulates a lot of browsing (opening and closing many sites and interacting with them in non-trivial ways).  At the end we could measure various things, such a memory usage, garbage and cycle collection time, and we could set targets to reduce those.  For mobile, the current MemShrink process probably doesn’t need to change that much, though more profiling on mobile devices would be good.

Personally, I’ve been spreading myself thinly over a lot of MemShrink bugs.  In particular, I try to push them along and not let them stall by doing things like trying to reproduce them, asking questions, etc.  I’ve been feeling lately like it would be a better use of my time to do less of that and instead dig deeply into a particular area.  I thought about working on making JS script compilation lazy, but now I’ve decided instead to focus primarily on improving the measurements in about:memory, in particular, reducing the size of “heap-unclassified” by improving existing memory reporters and adding new ones. I’ve decided this because it’s an area where I have expertise, clear ideas on how to make progress, and tools to help me.  Plus it’s important;  we can’t make improvements without measurements, and about:memory is the best memory measurement tool we have.  Hopefully other people agree that this is important to work on :)

SpiderMonkey is on a diet

One thing I’ve learnt while working for Mozilla is that a web browser can be characterized as a JavaScript execution environment that happens to have some multimedia capabilities.  In particular, if you look at Firefox’s about:memory page, the JS engine is very often the component responsible for consuming the most memory.

Consider the following snapshot from about:memory of the memory used by a single JavaScript compartment.

about:memory snapshot

(For those of you who have looked at about:memory before, some of those entries may look unfamiliar, because I landed a patch to refine the JS memory reporters late last week.)

There is work underway to reduce many of the entries in that snapshot.  SpiderMonkey is on a diet.

Objects

Objects are the primary data structure used in JS programs;  after all, it is an object-oriented language.  Inside SpiderMonkey, each object is represented by a JSObject, which holds basic information, and possibly a slots array, which holds the object’s properties. The memory consumption for all JSObjects is measured by the “gc-heap/objects/non-function” and “gc-heap/objects/function” entries in about:memory, and the slots arrays are measured by the “object-slots” entries.

The size of a non-function JSObject is currently 40 bytes on 32-bit platforms and 72 bytes on 64-bit platforms.  Brian Hackett is working to reduce that to 16 bytes and 32 bytes respectively. Function JSObjects are a little larger, being (internally) a sub-class of JSObject called JSFunction.  JSFunctions will therefore benefit from the shrinking of JSObject, and Brian is slimming down the function-specific parts as well.  In fact, these changes are complete in the JaegerMonkey repository, and will likely be merged into mozilla-central early in the Firefox 11 development period.

As for the slots arrays, they are currently arrays of “fatvals” A fatval is a 64-bit internal representation that can hold any JS value — number, object, string, whatever.  (See here for details, scroll down to “Mozilla’s New JavaScript Value Representation”;  the original blog entry is apparently no longer available).  64-bits per entry is overkill if you know, for example, that you have an array full entirely of integers that could fit into 32 bits.  Luke Wagner and Brian Hackett have been discussing a specialized representation to take advantage of such cases.  Variations on this idea have been tried twice before and failed, but perhaps SpiderMonkey’s new type inference support will provide the right infrastructure for it to happen.

Shapes

There are a number of data structures within SpiderMonkey dedicated to making object property accesses fast.  The most important of these are Shapes.  Each Shape corresponds to a particular property that is present in one or more JS objects.  Furthermore, Shapes are linked into linear sequences called “shape lineages”, which describe object layouts.  Some shape lineages are shared and live in “property trees”.  Other shape lineages are unshared and belong to a single JS object;  these are “in dictionary mode”.

The “shapes/tree” and “shapes/dict” entries in about:memory measure the memory consumption for all Shapes.  Shapes of both kinds are the same size;  currently they are 40 bytes on 32-bit platforms and 64 bytes on 64-bit platforms.  But Brian Hackett has also been taking a hatchet to Shape, reducing them to 24 bytes and 40 bytes respectively.  This has required the creation of a new auxiliary BaseShape type, but there should be many fewer BaseShapes than there are Shapes.  This change will also increase the number of Shapes, but should result in a space saving overall.

SpiderMonkey often has to search shape lineages, and for lineages that are hot it creates an auxiliary hash table, called a “property table”, that makes lookups faster.  The “shapes-extra/tree-tables” and “shapes-extra/dict-tables” entries in about:memory measure these tables.  Last Friday I landed a patch that avoids building these tables if they only have a few items in them;  in that case a linear search is just as good.  This reduced the amount of memory consumed by property tables by about 20%.

I mentioned that many Shapes are in property trees.  These are N-ary trees, but most Shapes in them have zero or one child;  only a small fraction have more than that, but the maximum N can be hundreds or even thousands.  So there’s a long-standing space optimization where each shape contains (via a union) a single Shape pointer which is used if it has zero or one child.  But if the number of children increases to 2 or more, this is changed into a pointer to a hash table, which contains pointers to the N children.  Until recently, if a Shape had a child deleted and that reduced the number of children from 2 to 1, it wouldn’t be converted from the hash form back to the single-pointer.  I changed this last Friday.  I also reduced the minimum size of these hash tables from 16 to 4, which saves a lot of space because most of them only have 2 or 3 entries.  These two changes together reduced the size of the “shapes-extra/tree-shape-kids” entry in about:memory by roughly 30–50%.

Scripts

Internally, a JSScript represents (more or less) the code of a JS function, including things like the internal bytecode that SpiderMonkey generates for it.  The memory used by JSScripts is measured by the “gc-heap/scripts” and “script-data” entries in about:memory.

Luke Wagner did some measurements recently that showed that most (70–80%) JSScripts created in the browser are never run.  In hindsight, this isn’t so surprising — many websites load libraries like jQuery but only use a fraction of the functions in those libraries.  It wouldn’t be easy, but if SpiderMonkey could be changed to generate bytecode for scripts lazily, it could reduce “script-data” memory usage by 60–70%, as well as shaving non-trivial amounts of time when rendering pages.

Trace JIT

TraceMonkey is SpiderMonkey’s original JIT compiler, which was introduced in Firefox 3.5.  Its memory consumption is measured by the “tjit-*” entries in about:memory.

With the improvements that type inference made to JaegerMonkey, TraceMonkey simply isn’t needed any more.  Furthermore, it’s a big hairball that few if any JS team members will be sad to say goodbye to.  (js/src/jstracer.cpp alone is over 17,000 lines and over half a megabyte of code!)

TraceMonkey was turned off for web content JS code when type inference landed.  And then it was turned off for chrome code.  And now it is not even built by default.  (The about:memory snapshot above was from a build just before it was turned off.)  And it will be removed entirely early in the Firefox 11 development period.

As well as saving memory for trace JIT code and data (including the wasteful ballast hack required to avoid OOM crashes in Nanojit, ugh), removing all that code will significantly shrink the size of Firefox’s code.  David Anderson told me the binary of the standalone JS shell is about 0.5MB smaller with the trace JIT removed.

Method JIT

JaegerMonkey is SpiderMonkey’s second JIT compiler, which was introduced in Firefox 4.0.  Its memory consumption is measured by the “mjit-code/*” and “mjit-data” entries in about:memory.

JaegerMonkey generates a lot of code.  This situation will hopefully improve with the introduction of IonMonkey, which is SpiderMonkey’s third JIT compiler.  IonMonkey is still in early development and won’t be integrated for some time, but it should generate code that is not only much faster, but much smaller.

GC HEAP

There is a great deal of work being done on the JS garbage collector, by Bill McCloskey, Chris Leary, Terrence Cole, and others.  I’ll just point out two long-term goals that should reduce memory consumption significantly.

First, the JS heap currently has a great deal of wasted space due to fragmentation, i.e. intermingling of used and unused memory.  Once moving GC — i.e. the ability to move things on the heap — is implemented, it will pave the way for a compacting GC, which is one that can move live things that are intermingled with unused memory into contiguous chunks of memory.  This is a challenging goal, especially given Firefox’s high level of interaction between JS and C++ code (because moving C++ objects is not feasible), but one that could result in very large savings, greatly reducing the “gc-heap/arena/unused” and “gc-heap-chunk-*-unused” measurements in about:memory.

Second, a moving GC is a prerequisite for a generational GC, which allocates new things in a small chunk of memory called a “nursery”.  The nursery is garbage-collected frequently (this is cheap because it’s small), and objects in the nursery that survive a collection are promoted to a “tenured generation”.  Generational GC is a win because in practice the majority of things allocated die quickly and are not promoted to the tenured generation.  This means the heap will grow more slowly.

Is that all?

It’s all I can think of right now.  If I’ve missed anything, please add details in the comments.

There’s an incredible amount of work being done on SpiderMonkey at the moment, and a lot of it will help reduce Firefox’s memory consumption.  I can’t wait to see what SpiderMonkey looks like in 6 months!

MemShrink progress, week 19

Good News

Peter Van der Beken fixed a leak caused by chrome code that injects a function into pages.  This was a MemShrink P1 bug.  The commentary in the bug is confusing, but this may have been affecting numerous add-ons including Firebug. (Bug 669730 is open for tracking leaks in Firebug;  it hasn’t yet been confirmed whether this fix has helped Firebug.)

Andrew McCreight rewrote JS_DumpHeap so that it dumps the complete JS object graph.  This was a MemShrink P1 bug because it’s an important piece of infrastructure for writing leak detection tools.

Justin Lebar fixed the measurement of RSS on Mac.  This was a MemShrink P1 bug because it prevented us from enabling jemalloc on Mac 10.5 machines.

Brian Hackett tweaked the JS engine so that more method JIT code can be discarded periodically, particularly chrome code in system compartments.

Oleg Romashin avoided using transparent layers in Fennec remote offscreen viewports.  This saves 5MB of memory in the active tab.

I added a memory reporter for the startup cache.  This is often around 1MB of memory, but it’s allocated via mmap/VirtualAlloc and so doesn’t change the “heap-unclassified” number in about:memory.

Boris Zbarsky fixed a small leak involving CSS transforms.

Bad News

It appears a bad memory regression has occurred in the past week or so.  Several people have reported multi-second pauses caused by garbage collection and cycle collection.  The problem only seems to strike when many tabs are open and/or the browser has been running for multiple days.  This needs investigation;  if you are experiencing similar problems please report in the bug.  Reliable steps to reproduce this bug will be invaluable.  If you turn on javascript.options.mem.log in about:config you can see when GCs and CCs occur and how long they take, which is helpful for diagnosis.

Bug Counts

The current bug counts are as follows.

  • P1: 35 (-5, +3)
  • P2: 113 (-3, +8)
  • P3: 54 (-0, +1)
  • Unprioritized: 4 (-5, +3)

I want to point out this bug, which presents an idea to help hunt down reproducible leaks that occur when users have multiple add-ons present.  This is important because many of the leaks reported by users recently are due to add-ons, but often the reporter has many add-ons installed which makes finding the culprit painful.  The goal is to write a Firefox add-on that selectively disables installed add-ons, so that a user can bisect them to discover which add-on is responsible for the leak.  This bisecting process is something that people can do manually, but an add-on that automates the process would make things easier and less error-prone.  Mercurial’s ‘hg bisect’ command would serve as a useful comparison.  Ehsan Akhgari has volunteered to mentor anyone who would like to try to implement this.

 

MemShrink progress, week 9

Firefox 8 graduated to the Aurora channel this week, and the development period for what will become Firefox 9 began.  Lots of MemShrink activity happened this week, and I think all the changes listed below will make it into Firefox 8.

Avoiding Wasted Memory

I have blogged previously about memory wasted by “clownshoes” bugs.   Ed Morley found a webpage that resulted in 700MB of memory being wasted by the PLArena clownshoes bug.  Basically, on platforms where jemalloc is used (Windows, Linux), half the memory allocated by nsPresArena (which is built on top of PLArena) was wasted.  (On Mac the waste was 11%, because the Mac allocator rounds up less aggressively than jemalloc).

Fixing this problem properly for all PLArenas takes time because it requires changes to NSPR, so I made a spot-fix for the nsPresArena case.  This is a particularly big win on very large pages, but it saves around 3MB even on Gmail. This spot-fix has been granted beta approval and so will, barring disaster, make it into Firefox 7.

A Firefox Nightly user did some measurements with different browsers on the problematic page:

  • Firefox 8.0a1 before patch: 2.0 GB
  • Firefox 8.0a1 after patch: 1.3 GB
  • Latest Chrome canary build and dev (15.0.849.0): 1.1GB
  • Webkit2Process of Safari 5.1: 1.05 GB
  • Internet Explorer 9.0.2: 838 MB
  • Latest Opera Next 12.00: 727 MB

So this fix gets Firefox within spitting distance of other browsers, which is good!

In other developments related to avoiding wasted memory:

  • Luke Wagner discovered that, on typical websites, most JSScripts are byte-compiled but never run.  A JSScript roughly corresponds to a JavaScript function.  In hindsight, it’s not such a surprising result — Firefox byte-compiles all loaded JavaScript code, and you can imagine lots of websites use libraries like jQuery but only use a small fraction of the functions in the library.  Making byte-compilation lazy could potentially save MBs of memory per compartment.  But that will require some non-trivial reworking of the JS engine, and so is unlikely to happen in the short-term.
  • Kyle Huey avoided a small amount (~100KB per browser process) of waste due to rounding up in XPT arenas.

Improving about:memory

I made some progress on a Valgrind tool to help identify the memory that is currently reported only by the “heap-unclassified” bucket in about:memory.  It’s called “DMD”, short for “Dark Matter Detector”.  It’s in early stages and I still need to teach it about most of Firefox’s memory reporters, but it’s already spitting out useful data, which led to me and Ehsan Akhgari landing memory reporters for the JS atom table and the Hunspell spell checker.  We also now have some insight (here and here) about memory usage for very large pages.

Mounir Lamouri turned on the memory reporter for the DOM that he’s been working on for some time.  This shows up under “dom” in about:memory.  There are still some cases that require handling;  you can follow the progress of these here.

Andrew McCreight replaced about:memory’s buttons so you can force a cycle collection without also forcing a garbage collection, which may be useful in hunting down certain problems.

Finally, Sander van Veen added the existing “js-compartments-user” and “js-compartments-system” to the statistics collected by telemetry (his first landed patch!), and I did likewise for the “storage/sqlite” reporter.  I also added a new “tjit-data/trace-monitor” memory reporter that accounts for some of the memory used by TraceMonkey.

Miscellaneous

Igor Bukanov tweaked the handling of empty chunks by the JavaScript garbage collector.  That sounds boring until you see the results on Gregor Wagner’s 150-tab stress test: resident memory usage dropped 9.5% with all 150 tabs open, and dropped by 27% after all those tabs were closed.

Brian Hackett fixed a memory leak in type inference, which gets it one step closer to being ready to land.

Christian Höltje fixed a leak in his “It’s All Text” add-on that was causing zombie compartments.  This fix will be in version 1.6.0, which is currently awaiting to receive AMO approval, but can be obtained here in the meantime.  This fix and last week’s fix of a memory leak in LastPass are very encouraging — per-compartment reporters in about:memory have, for the first time, given add-on developers a reasonable tool for identifying memory leaks.  I hope we can continue to improve the situation here.  Several people have asked me for documentation on how to avoid memory leaks in add-ons.  I’m not the person to write that guide (I’m not a Gecko expert and I know almost nothing about add-ons) but hopefully someone else can step up to the plate.

Bug counts

Here’s the change in MemShrink bug counts.

  • P1: 30 (-0, +1)
  • P2: 64 (-4, +6)
  • P3: 36 (-5, +0)
  • Unprioritized: 1 (-2, +1)

Good progress on P3 bugs, but they’re the least important ones.  Other than that, new bugs are still being reported faster than they’re being fixed.  If you want to help but don’t know where to start, feel free to email me or ping me on IRC and I’ll do my best to help get you involved.