The benefits of reducing memory consumption

TL;DR: Any single change that reduces Firefox’s memory consumption can affect Firefox’s speed, stability and reputation in a variety of ways, some of which are non-obvious.  Some examples illustrate this.

The MemShrink wiki page starts with the following text.

MemShrink is a project that aims to reduce Firefox’s memory consumption. There are three potential benefits.  Speed. […] Stability. […] Reputation.”

I want to dig more deeply into these benefits and the question of what it means to “reduce Firefox’s memory consumption”, because there are some subtleties involved.  In what follows I will use the term “MemShrink optimization” to refer to any change that reduces Firefox’s memory consumption.

Speed

People tend to associate low memory consumption with speed.  However, time/space trade-offs abound in programming, and an explicit goal of MemShrink is to not slow Firefox down — the wiki page says:

Changes that reduce memory consumption but make Firefox slower are not desirable.

There are several ways that MemShrink optimizations can improve performance.

Paging

The case that people probably think of first is paging.  If physical memory fills up and the machine needs to start paging, i.e. evicting virtual memory pages to disk space, it can be catastrophic for performance.  This is because disk accesses are many thousands of times slower than RAM accesses.

However, some MemShrink optimizations are far more likely to affect paging than others.  The key idea here is that of the working set size — what’s important is not the total amount of physical or virtual memory being used, but the fraction of that memory that is touched frequently.  For example, consider two programs that allocate and use a 1GB array.  The first one touches pages within the array at random.  The second one touches every page once and then touches the first page many times.  The second program will obviously page much less than the first if the system’s physical memory fills up.

The consequence of this is that a change that reduces the size of data structures that are accessed frequently is much more likely to reduce paging than a change that reduces the size of data structures that are accessed rarely.  Note that this is counter-intuitive!  It’s natural to want to optimize data structures that are wasteful of space, but “wasteful of space” often means “hardly touched” and so such optimizations don’t have much effect on paging.

Measuring the working set size of a complex program like a web browser is actually rather difficult, which means that gauging the impact of a change on paging is also difficult.  Julian Seward’s virtual memory profiler offers one possible way.  Another complication is that results vary greatly between machines.  If you are running Firefox on a machine with 16GB of RAM, it’s likely that no change will affect paging, because Firefox is probably never paging in the first place.  If you are on a netbook with 1GB of RAM, the story is obviously different.  Also, the effects can vary between different operating systems.

Cache pressure

Some MemShrink optimizations can also reduce cache pressure.  For example, a change that makes a struct smaller would allow more of them to fit into a cache line.  Like paging, these effects are very difficult to quantify, and changes that affect hot structures are more likely to reduce cache pressure significantly and improve performance.

Structure traversals

Sometimes large data structures must be traversed, and reducing the number of elements in the data structure can reduce that traversal time.  The obvious case for Firefox is the JavaScript heap — the garbage collector and cycle collector frequently traverse it, and so any change that causes dead objects to accumulate more slowly will reduce their traversal times.

Only a small fraction of MemShrink optimizations will speed up structure traversals.

Stability

If Firefox (or any program) uses too much memory, it can lead to aborts and crashes.  These are sometimes called “OOMs” (out of memory). There are two main kinds of OOM:  those involving virtual memory, and those involving physical memory.

Virtual OOMs

A “virtual OOM” occurs when the virtual address space fills up and Firefox simply cannot refer to any more memory.  This is mostly a problem on Windows, where Firefox is distributed as a 32-bit application, and so it can only address 2GB or 4GB of memory (the addressable amount depends on the OS configuration).  This is true even if you have more than 4GB of RAM.  In contrast, Mac OS X and Linux builds of Firefox are 64-bit and so virtual memory exhaustion is essentially impossible because the address space is massively larger.

(I don’t want to get distracted by the question of why Firefox is a 32-bit application on Windows.  I’ll just mention that (a) many Windows users are still running 32-bit versions of Windows that cannot run 64-bit applications, and (b) Mozilla does 64-bit Windows builds for testing purposes.  Detailed discussions of the pros and cons of 64-bit builds can be read here and here.)

The vast majority of MemShrink optimizations will reduce the amount of virtual memory consumed.  (The only counter-examples I can think of involve deliberately evicting pages from physical memory.  E.g. see the example of the GC decommitting change discussed below.)  And any such change will obviously reduce the number of virtual OOMs.  Furthermore, the effect of any reduction is obvious and straightforward — a change that reduces the virtual memory consumption by 100MB on a particular machine and workload is twice as good as one that reduces it by 50MB.  Of course, any improvement will only be noticed by those who experience virtual OOMs, which typically is people who have 100s of tabs open at once.  (It may come as a surprise, but some people have that many tabs open regularly.)

Physical OOMs

A “physical OOM” occurs when physical memory (and any additional backing storage such as swap space on disk) fills up.  This is mostly a problem on low-end devices such as smartphones and netbooks, which typically have small amounts of RAM and may not have any swap space.

The situation for physical memory is similar to that for virtual memory:  almost any MemShrink optimization will reduce Firefox’s physical memory consumption.  (One exception is that it’s possible for a memory allocation to consume virtual memory but not physical memory if it’s never accessed;  more about this in the examples section below.)  And any reduction in physical memory consumption will in turn reduce the number of physical OOMs.  Finally, the effects are again obvious and straightforward — a 100MB reduction is twice as good as a 50MB reduction.

Reputation

Finally, we have reputation.  The obvious effect here is that if MemShrink optimizations cause Firefox to become faster and more stable over time, people’s opinion of Firefox will rise, either because their own experience improves, or they hear that other people’s experience improves.

But I want to highlight a less obvious aspect of reputation.  People often gauge Firefox’s memory consumption by looking at a utility such as the Task Manager (on Windows) or ‘top’ (on Mac/Linux).  Interpreting the numbers from these utilities is rather difficult — there are multiple metrics and all sorts of subtleties involved.  (See this Stack Overflow post for evidence of the complexities and how easy it is to get things wrong.)  In fact, in my opinion, the subtleties are so great that people should almost never look at these numbers and instead focus on metrics that are influenced by memory consumption but which they can observe directly as users, i.e. speed and crash rate… but that’s a discussion for another time.

Nonetheless, a non-trivial number of people judge Firefox on this metric.  Imagine a change that caused Firefox’s numbers in these utilities to drop but had no other observable effect.  (Such a change may be impossible in practice, but that doesn’t matter in this thought experiment.)  One thing that has consistently surprised me is that some people view memory consumption as something approaching a moral issue:  low memory consumption is virtuous and high memory consumption is sinful.  As a result, this hypothetical change would improve Firefox’s reputation, rightly or wrongly, for the better.

Let’s call this aspect of Firefox’s reputation the “reputation-by-measurement”.  I suspect the most important metric for reputation-by-measurement is the “private bytes” reported by the Windows Task Manager, because that’s what people seem to most often look at.  Private bytes measures the virtual memory of a process that is not shared with any other process.  It’s my educated guess that in Firefox’s case that the amount of shared memory isn’t that high, and so the situation is similar to virtual OOMs above — just about any change that reduces the amount of virtual memory will reduce the private bytes by the same amount, and in terms of reputation-by-measurement, a 100MB reduction is twice as good as a 50MB reduction.

Examples

Some examples help bring this discussion together.  Consider bug 609905, which removed a 512KB block of memory that was found to be allocated but never accessed.  (This occurred because some code that used that block was removed but the allocation wasn’t removed at the same time.)  What were the benefits of this change?

• The 512KB never would have been in the working set, so performance would not have been affected.
• Virtual memory consumption would have dropped by 512KB, slightly reducing the likelihood of virtual OOMs.
• Physical memory consumption probably didn’t change — because the block was never accessed, it probably never made it into physical memory.
• Private bytes would have dropped by 512KB, slightly improving reputation-by-measurement.

Now consider bug 670596, which made the JavaScript garbage collector decommit (i.e. remove from physical memory and backing storage) 1MB heap chunks that are unused.  What were the benefits of this change?

• Performance may have improved slightly due to reduced paging, on machines where paging happens.  Those chunks are clearly not in the working set when they are decommitted, but if paging occurs, the pre-emptive removal of some pages from physical memory may prevent the OS from having to evict some other pages, some of which might have been in the working set.
• Virtual memory consumption would not have changed at all, because decommitted memory still takes up address space.
• Physical memory consumption would have dropped by the full decommit amount — 10s or even 100s of MBs in many cases when decommitting is triggered — significantly reducing the likelihood of physical OOMs.
• Private bytes would not have changed, leaving reputation-by-measurement unaffected. [Update: Justin Lebar queried this. This page indicates that decommitting memory does reduce private bytes, which means that this change would have improved reputation-by-measurement.]

Another interesting one is bug 676457.  It fixed a problem where PLArenaPool was requesting lots of allocations of ~4,128 bytes.  jemalloc rounded these requests up to 8,192 bytes, so almost 50% of each 8,192 byte block was wasted, and there could be many of these blocks.  The patch fixed this by reducing the requests to 4,096 bytes which is a power-of-two and not rounded up (and also usually the size of an OS virtual memory page).  What were the benefits of this change?

• Performance may have improved due to less paging, because the working set size may have dropped.  The size of the effect depends how often the final 32 used bytes of each chunk — those that spilled onto a second page — are accessed.  For at least some of the blocks those 32 bytes would never be touched.
• Virtual memory consumption dropped significantly, reducing the likelihood of virtual OOMs.
• Physical memory consumption may have dropped, but it’s not clear by how much.  In cases where the extra 32 bytes are never accessed, the second page might not have ever taken up physical memory.
• Private bytes would have dropped by the same amount as virtual memory, improving reputation-by-measurement.

Finally, let’s think about a compacting, generational garbage collector, something that is being worked on by the JS team at the moment.

• Performance improves for three reasons.  First, paging is reduced because of the generational behaviour:  much of the JS engine activity occurs in the nursery, which is small;  in other words, the memory activity is concentrated within a smaller part of the working set.  Second, paging is further reduced because of the compaction: this reduces fragmentation within pages in the tenured heap, reducing the total working set size.  Third, the tenured heap grows more slowly because of the generational behaviour: many objects are collected earlier (in the nursery) than they would be with a non-generational collector, which means that structure traversals done by the garbage collector (during full-heap collections) and cycle collector are faster.
• Virtual memory consumption drops in two ways.  First, the compaction minimizes waste due to fragmentation.  Second, the heap grows more slowly.
• Physical memory consumption drops for the same two reasons.
• Private bytes also drops for the same two reasons.

A virtuous change indeed.

Conclusion

Reducing Firefox’s memory consumption is a good thing, and it has the following benefits.

• It can improve speed, due to less paging, fewer cache misses, and faster structure traversals.  These changes are likely to be noticed more by users on lower-end machines.
• It improves stability by reducing virtual OOM aborts, which mostly helps heavy tab users on Windows.  It also improves stability by reducing physical OOM aborts, which mostly affects heavy-ish tab users on small devices like smartphones and netbooks.
• It improves reputation among those whose browsing experience is improved by the above changes, and also among users who make judgments according to memory measurements with utilities like the Windows Task Manager.

Furthermore, when discussing MemShrink optimizations, it’s a good idea to describe improvements in these terms.  For example, instead of saying “this change reduces memory consumption”, one could say “this change reduces physical and virtual memory consumption and may reduce paging”.  I will endeavour to do this myself from now on.