about:memory has some wonderful characteristics:\u00a0 it provides literally thousands of measurements;\u00a0 it’s available in ordinary release builds;\u00a0 and it’s trivial to run (just type “about:memory” in the address bar).\u00a0 This means that non-expert users can easily provide developers with detailed measurements if they are having problems.<\/p>\n However, about:memory has two shortcomings.\u00a0 First, it simply visualizes the data provided by the memory reporter<\/em> infrastructure.\u00a0 The coverage provided by this infrastructure is good, but there are still some gaps.\u00a0 These gaps manifest primarily in the “heap-unclassified” number in about:memory, which represents all the heap allocations that the memory reporters didn’t cover.\u00a0 Unfortunately, about:memory can provide zero insight into what is within “heap-unclassified”.<\/p>\n Second, it’s hard to verify.\u00a0 There’s no obvious way to tell if the numbers it gives are accurate (with a few exceptions;\u00a0 e.g. negative numbers are obviously wrong).\u00a0 We have established good practices for writing memory reporters (measure sizes, don’t compute then;\u00a0 traverse data structures rather than maintaining size counters<\/a>) that prevent most errors, but it’s still quite easy to accidentally count a block of memory twice.<\/p>\n This is where DMD comes in.\u00a0 It helps with both the heap-unclassified and double-counting problems.<\/p>\n I wrote the first version of DMD<\/a> over a year ago.\u00a0 “DMD” is short for “dark matter detector”, because “heap-unclassified” memory is sometimes jokingly called “dark matter”.<\/p>\n DMD works by intercepting all calls to malloc\/free\/etc.\u00a0 This lets DMD track extra information about every heap block, such as where it was allocated.\u00a0 Furthermore, DMD has hooks into the memory reporters (via functions created with the NS_MEMORY_REPORTER_MALLOC_SIZEOF_FUN macro, for those who are interested) so it knows when any heap block is measured by a memory reporter.<\/p>\n With that in place, DMD knows how many times each heap block has been reported.\u00a0 So, after running the memory reporters, we can up each block into one of the following three groups.<\/p>\n DMD presents its results in a sorted fashion that lists the unreported and twice-reported cases that are more important first.<\/p>\n This first version of DMD was implemented as a Valgrind tool, and what’s more, it required a special, patched build of Valgrind to run.\u00a0 This meant it was difficult to set up, ran very slowly, and could only be used on Linux and Mac.\u00a0 Despite these shortcomings, it has proved itself extremely useful, helping us get “heap-unclassified” down greatly (on my Linux desktop machine it’s typically between 8 and 12%) and identifying several cases of double-counting.<\/p>\n Some time after I wrote the first version of DMD, I realized that all it needed to work was the ability to intercept malloc\/free\/etc. and to obtain stack traces.\u00a0 Valgrind was overkill for these purposes — it’s capable of supporting much more invasive tools — and, furthermore, there were existing tools (e.g. trace-malloc) in the Mozilla codebase that did exactly these things.\u00a0 In other words, it would be possible to build a new version of DMD<\/a> that integrated directly into the browser.<\/p>\n Work on this new version stalled for a while, because the old one was working well enough.\u00a0 But then B2G’s Operation Slim Fast<\/a> started, and it quickly became obvious that “heap-unclassified” on B2G<\/a> was typically much higher than it was on desktop.\u00a0 This is primarily because B2G has lots of small processes, and so various unmeasured things that weren’t a big deal on desktop became much more significant on B2G.\u00a0 And DMD version 1 doesn’t work on B2G devices.<\/p>\n So that provided the impetus to complete the new version.\u00a0 Mike Hommey finished his new replace-malloc infrastructure<\/a> recently, which helped greatly, and the new version of DMD landed on mozilla-central last week.\u00a0 The old version will soon be retired<\/a>.<\/p>\n![\"about:memory](\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2012\/12\/am.png\" "\\"about:memory")
<\/a><\/p>\nDMD version 1<\/h3>\n
\n
DMD Version 2<\/h3>\n