{"id":771,"date":"2011-05-23T15:03:32","date_gmt":"2011-05-23T04:03:32","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=771"},"modified":"2011-05-23T16:34:03","modified_gmt":"2011-05-23T05:34:03","slug":"a-better-aboutmemory-stage-1-75","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2011\/05\/23\/a-better-aboutmemory-stage-1-75\/","title":{"rendered":"A better about:memory: stage 1.75"},"content":{"rendered":"<p>I just landed <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=657327\">bug 657327<\/a>, which makes about:memory simpler and more useful.\u00a0 To understand the change, let&#8217;s look at what about:memory looked like before the change landed.<\/p>\n<p><a href=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-o.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-772\" title=\"aboutmemory-o\" src=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-o.png\" alt=\"Old about:memory screenshot\" width=\"383\" height=\"410\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-o.png 383w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-o-280x300.png 280w\" sizes=\"(max-width: 383px) 100vw, 383px\" \/><\/a><\/p>\n<p>The first thing to look at is the &#8220;mapped&#8221; entry at the top of the &#8220;Mapped Memory&#8221; tree.\u00a0 It was meant to measure the total memory (both private and shared) mapped by the process.\u00a0 But there were a couple of problems with it:<\/p>\n<ul>\n<li>On Windows, it only measured the private bytes.\u00a0 There&#8217;s no easy way I know of to measure the shared bytes as well.\u00a0 This could lead to negative numbers in the output (<a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=655642\">bug 655642<\/a>).<\/li>\n<li>On Mac, the number includes an enormous amount of shared memory mapped.\u00a0 If you have a Mac, run &#8216;top&#8217; and look at the VSIZE column.\u00a0 Almost every process has a value of 2GB or greater.\u00a0 So the &#8220;mapped&#8221; value is really high, which looks bad, even though it&#8217;s not Firefox&#8217;s fault.<\/li>\n<li>Even on Linux, where the amount of shared memory is smaller and so the &#8220;mapped&#8221; number is reasonable, it&#8217;s still not that useful, because it includes memory mappings like code and data segments that aren&#8217;t that interesting.<\/li>\n<\/ul>\n<p>So, in summary, the very first number shown on about:memory was (a) incorrect on Windows, (b) misleadingly inflated on Mac, and (c) not much use on Linux.<\/p>\n<p>The other thing to notice about the old about:memory is that there are two trees, &#8220;Mapped Memory&#8221; and &#8220;Used Heap Memory&#8221;.\u00a0 For the purposes of this discussion, memory usage can be split into four groups.<\/p>\n<ol>\n<li>Explicitly allocated heap memory.\u00a0 This is heap memory requested by Firefox through the heap allocation functions like malloc, calloc, realloc, and C++&#8217;s &#8216;operator new&#8217;.<\/li>\n<li>Implicitly allocated heap memory.\u00a0 This is heap memory that has been freed by Firefox through the heap deallocate functions like free and C++&#8217;s &#8216;operator delete&#8217;, but which the heap allocator (e.g. jemalloc) has not yet handed back to the OS, for whatever reason.<\/li>\n<li>Explicitly allocated mapped memory.\u00a0 This is memory requested by Firefox through OS-level allocation functions like mmap (on Linux and Mac), VirtualAlloc (on Windows), and vm_allocate (on Mac).<\/li>\n<li>Implicitly allocated mapped memory.\u00a0 This is memory allocated by the OS that hasn&#8217;t been explicitly requested by Firefox.\u00a0 It includes code and data segments (which are created when the executable and shared libraries are loaded) and thread stacks (which are created when threads are created).<\/li>\n<\/ol>\n<p>In the old about:memory, 1 is shown in the &#8220;Used Heap Memory&#8221; tree, and 2, 3 and 4 are shown in the &#8220;Mapped Memory&#8221;.\u00a0 But it&#8217;s 1 and 3 that we&#8217;re most interested in, because that&#8217;s memory that has been explicitly requested (and not yet freed) by Firefox.\u00a0 That&#8217;s where most of the dynamic variation in memory usage occurs, and that&#8217;s where memory leaks occur.<\/p>\n<p>The new about:memory reflects this better.<\/p>\n<p><a href=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-773\" title=\"aboutmemory\" src=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory.png\" alt=\"New about:memory screenshot\" width=\"632\" height=\"733\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory.png 632w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-258x300.png 258w\" sizes=\"(max-width: 632px) 100vw, 632px\" \/><\/a><\/p>\n<p>It has a single tree which only includes explicit allocations, and which does not distinguish between heap-level allocations (e.g. malloc) and OS-level allocations (e.g. mmap);\u00a0 this shortens the output and reduces the amount of nesting in the tree.\u00a0 Implicit allocations (2 and 4 above) are still covered, but only in the less-prominent &#8220;Other Measurements&#8221; list (under &#8220;vsize&#8221; and &#8220;heap-unused&#8221;).\u00a0 And the &#8220;explicit&#8221; entry, the very first one, is now the single most interesting number on the page.\u00a0 (Thanks to Jesse Ruderman for suggesting that I merge the two trees and flatten the resulting tree.)<\/p>\n<p>One disadvantage of the new form is that some explicit OS-level allocations may not be accounted for.\u00a0 (The full heap is always accounted for, thankfully.)\u00a0 I&#8217;m in the process of adding more memory reporters for significant OS-level allocations (e.g. <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=546477\">bug 546477<\/a>).\u00a0 Fortunately there doesn&#8217;t seem to be many.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I just landed bug 657327, which makes about:memory simpler and more useful.\u00a0 To understand the change, let&#8217;s look at what about:memory looked like before the change landed. The first thing to look at is the &#8220;mapped&#8221; entry at the top of the &#8220;Mapped Memory&#8221; tree.\u00a0 It was meant to measure the total memory (both private [&hellip;]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4550,30,4544],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/771"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=771"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/771\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=771"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=771"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=771"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}