A better about:memory: stage 1.75

I just landed bug 657327, which makes about:memory simpler and more useful.  To understand the change, let’s look at what about:memory looked like before the change landed.

Old about:memory screenshot

The first thing to look at is the “mapped” entry at the top of the “Mapped Memory” tree.  It was meant to measure the total memory (both private and shared) mapped by the process.  But there were a couple of problems with it:

  • On Windows, it only measured the private bytes.  There’s no easy way I know of to measure the shared bytes as well.  This could lead to negative numbers in the output (bug 655642).
  • On Mac, the number includes an enormous amount of shared memory mapped.  If you have a Mac, run ‘top’ and look at the VSIZE column.  Almost every process has a value of 2GB or greater.  So the “mapped” value is really high, which looks bad, even though it’s not Firefox’s fault.
  • Even on Linux, where the amount of shared memory is smaller and so the “mapped” number is reasonable, it’s still not that useful, because it includes memory mappings like code and data segments that aren’t that interesting.

So, in summary, the very first number shown on about:memory was (a) incorrect on Windows, (b) misleadingly inflated on Mac, and (c) not much use on Linux.

The other thing to notice about the old about:memory is that there are two trees, “Mapped Memory” and “Used Heap Memory”.  For the purposes of this discussion, memory usage can be split into four groups.

  1. Explicitly allocated heap memory.  This is heap memory requested by Firefox through the heap allocation functions like malloc, calloc, realloc, and C++’s ‘operator new’.
  2. Implicitly allocated heap memory.  This is heap memory that has been freed by Firefox through the heap deallocate functions like free and C++’s ‘operator delete’, but which the heap allocator (e.g. jemalloc) has not yet handed back to the OS, for whatever reason.
  3. Explicitly allocated mapped memory.  This is memory requested by Firefox through OS-level allocation functions like mmap (on Linux and Mac), VirtualAlloc (on Windows), and vm_allocate (on Mac).
  4. Implicitly allocated mapped memory.  This is memory allocated by the OS that hasn’t been explicitly requested by Firefox.  It includes code and data segments (which are created when the executable and shared libraries are loaded) and thread stacks (which are created when threads are created).

In the old about:memory, 1 is shown in the “Used Heap Memory” tree, and 2, 3 and 4 are shown in the “Mapped Memory”.  But it’s 1 and 3 that we’re most interested in, because that’s memory that has been explicitly requested (and not yet freed) by Firefox.  That’s where most of the dynamic variation in memory usage occurs, and that’s where memory leaks occur.

The new about:memory reflects this better.

New about:memory screenshot

It has a single tree which only includes explicit allocations, and which does not distinguish between heap-level allocations (e.g. malloc) and OS-level allocations (e.g. mmap);  this shortens the output and reduces the amount of nesting in the tree.  Implicit allocations (2 and 4 above) are still covered, but only in the less-prominent “Other Measurements” list (under “vsize” and “heap-unused”).  And the “explicit” entry, the very first one, is now the single most interesting number on the page.  (Thanks to Jesse Ruderman for suggesting that I merge the two trees and flatten the resulting tree.)

One disadvantage of the new form is that some explicit OS-level allocations may not be accounted for.  (The full heap is always accounted for, thankfully.)  I’m in the process of adding more memory reporters for significant OS-level allocations (e.g. bug 546477).  Fortunately there doesn’t seem to be many.

3 Responses to A better about:memory: stage 1.75

  1. One interesting thing I’ve noticed is that out “heap committed” measurement corresponds pretty well to what the Mac Activity Monitor reports under “Real Mem”. An in my case that’s about 2GB right now, while “Heap Used” is only 1GB… I wonder how much of that is fragmentation.

  2. Please give us a tool to measure addon memory consumption… :/

  3. After using about:memory for a time I have a question. Is the 100% explicit the memory that Firefox should use if it was perfect with memory handling?

    I ask this because I see the explicit number move in the way I would expect when I open and close tabs. So my explicit is now at 356MB, my resident is 535MB and Windows claims it is 474MB. Chrome does this really well and you immediately get the memory back from a tab when you close it.