I recently landed a new version of DMD on mozilla-central. DMD is a tool helps us understand and thus reduce Firefox’s memory consumption. But in order to understand DMD, you first have to understand about:memory.
about:memory
The MemShrink project started about 18 months ago, and it has been very successful in reducing Firefox’s memory consumption. about:memory is MemShrink’s not-so-secret weapon when it comes to understanding Firefox’s memory consumption.
about:memory has some wonderful characteristics: it provides literally thousands of measurements; it’s available in ordinary release builds; and it’s trivial to run (just type “about:memory” in the address bar). This means that non-expert users can easily provide developers with detailed measurements if they are having problems.
However, about:memory has two shortcomings. First, it simply visualizes the data provided by the memory reporter infrastructure. The coverage provided by this infrastructure is good, but there are still some gaps. These gaps manifest primarily in the “heap-unclassified” number in about:memory, which represents all the heap allocations that the memory reporters didn’t cover. Unfortunately, about:memory can provide zero insight into what is within “heap-unclassified”.
Second, it’s hard to verify. There’s no obvious way to tell if the numbers it gives are accurate (with a few exceptions; e.g. negative numbers are obviously wrong). We have established good practices for writing memory reporters (measure sizes, don’t compute then; traverse data structures rather than maintaining size counters) that prevent most errors, but it’s still quite easy to accidentally count a block of memory twice.
This is where DMD comes in. It helps with both the heap-unclassified and double-counting problems.
DMD version 1
I wrote the first version of DMD over a year ago. “DMD” is short for “dark matter detector”, because “heap-unclassified” memory is sometimes jokingly called “dark matter”.
DMD works by intercepting all calls to malloc/free/etc. This lets DMD track extra information about every heap block, such as where it was allocated. Furthermore, DMD has hooks into the memory reporters (via functions created with the NS_MEMORY_REPORTER_MALLOC_SIZEOF_FUN macro, for those who are interested) so it knows when any heap block is measured by a memory reporter.
With that in place, DMD knows how many times each heap block has been reported. So, after running the memory reporters, we can up each block into one of the following three groups.
- Blocks that haven’t been reported indicate gaps in existing memory reporters. They contribute to “heap-unclassified”.
- Blocks that have been reported once are good.
- Blocks that have been reported twice or more indicate defects in existing memory reporters. They cause some measurements in about:memory to be too high, and “heap-unclassified” to be too low.
DMD presents its results in a sorted fashion that lists the unreported and twice-reported cases that are more important first.
This first version of DMD was implemented as a Valgrind tool, and what’s more, it required a special, patched build of Valgrind to run. This meant it was difficult to set up, ran very slowly, and could only be used on Linux and Mac. Despite these shortcomings, it has proved itself extremely useful, helping us get “heap-unclassified” down greatly (on my Linux desktop machine it’s typically between 8 and 12%) and identifying several cases of double-counting.
DMD Version 2
Some time after I wrote the first version of DMD, I realized that all it needed to work was the ability to intercept malloc/free/etc. and to obtain stack traces. Valgrind was overkill for these purposes — it’s capable of supporting much more invasive tools — and, furthermore, there were existing tools (e.g. trace-malloc) in the Mozilla codebase that did exactly these things. In other words, it would be possible to build a new version of DMD that integrated directly into the browser.
Work on this new version stalled for a while, because the old one was working well enough. But then B2G’s Operation Slim Fast started, and it quickly became obvious that “heap-unclassified” on B2G was typically much higher than it was on desktop. This is primarily because B2G has lots of small processes, and so various unmeasured things that weren’t a big deal on desktop became much more significant on B2G. And DMD version 1 doesn’t work on B2G devices.
So that provided the impetus to complete the new version. Mike Hommey finished his new replace-malloc infrastructure recently, which helped greatly, and the new version of DMD landed on mozilla-central last week. The old version will soon be retired.
DMD now works on Linux, Android, Mac, B2G, and is just shy of working on Windows (where Ehsan Akhgari has been making gradual progress). It’s also much faster, partly because it avoids the overhead of Valgrind, and partly because it now uses sampling. The sampling causes blocks smaller than a certain size (4 KiB by default, though you can change that) to be sampled, while blocks larger than that are measured accurately; this sacrifices a moderate amount of accuracy for a large amount of speed.
Try it
about:memory is wonderful, and DMD is how we make about:memory better. There are now detailed instructions on how to build DMD, run it, and interpret its output. It’s quite easy now, requiring only minor changes to the usual build and run steps, and several people have already done it successfully. Please try it out, and help us further improve about:memory.