{"id":671,"date":"2011-05-02T15:54:49","date_gmt":"2011-05-02T04:54:49","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=671"},"modified":"2011-05-12T14:11:08","modified_gmt":"2011-05-12T03:11:08","slug":"a-better-aboutmemory-stage-1","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2011\/05\/02\/a-better-aboutmemory-stage-1\/","title":{"rendered":"A better about:memory: stage 1"},"content":{"rendered":"<p>A while ago I wrote about <a href=\"http:\/\/blog.mozilla.org\/nnethercote\/2011\/02\/09\/a-vision-for-better-memory-profiling-with-aboutmemory\/\">some ideas<\/a> I had for improving <a href=\"about:memory\">about:memory<\/a>.\u00a0 <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=633653\">Bug 653633 <\/a>has been tracking the first stage of this (and it now also has a <a href=\"https:\/\/wiki.mozilla.org\/Platform\/Features\/RevampAboutMemory\">feature page<\/a>) and it only needs to pass super-review to be ready for landing.<\/p>\n<p>Here&#8217;s what the old about:memory looks like.<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-old.png\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-673 aligncenter\" title=\"aboutmemory-old\" src=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-old.png\" alt=\"Old about:memory output\" width=\"336\" height=\"854\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-old.png 336w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-old-118x300.png 118w\" sizes=\"(max-width: 336px) 100vw, 336px\" \/><\/a><\/p>\n<p>Some things to note:<\/p>\n<ul>\n<li>The &#8220;Memory mapped&#8221; and &#8220;Memory in use&#8221; numbers are for the heap, not the entire process.<\/li>\n<li>The &#8220;Memory mapped&#8221; and &#8220;malloc\/mapped&#8221; numbers are the same, as are the &#8220;Memory in use&#8221; and &#8220;malloc\/allocated&#8221; numbers (modulo tiny differences due to memory allocations that occur while generating the page).<\/li>\n<li>It&#8217;s a big list without much information how the different entries relate.\u00a0 If you hover over the names you get a tool-tip that explains a bit more, but it&#8217;s often not much of a help.<\/li>\n<li>Something not obvious from the above screenshot is that if you cut and paste the output into a text box it looks horrible.\u00a0 Eg. see <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=633653#c0\">this Bugzilla comment<\/a>.\u00a0 This makes it hard to use in Bugzilla reports.<\/li>\n<\/ul>\n<p>Here&#8217;s what the new about:memory looks like:<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-new.png\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-674 aligncenter\" title=\"aboutmemory-new\" src=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-new.png\" alt=\"New about:memory output\" width=\"405\" height=\"1184\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-new.png 405w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-new-350x1024.png 350w\" sizes=\"(max-width: 405px) 100vw, 405px\" \/><\/a><\/p>\n<p>Things to note:<\/p>\n<ul>\n<li>Most of the output use a tree-based representation which shows the hierarchy of the memory breakdown.\u00a0 I.e. overall use is broken into multiple chunks, and each of those chunks are broken into sub-chunks, and so on.\u00a0 This makes it much easier to see where the bulk of the memory usage is occurring.<\/li>\n<li>The use of percentages helps with this too.<\/li>\n<li>Sizes are reported in MB, not bytes.\u00a0 (I wanted to use &#8220;MiB&#8221; instead of &#8220;MB&#8221;, but was discouraged by the prior holy war about this issue that <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=106618\">bug 106618<\/a> documents!)<\/li>\n<li>The &#8220;Mapped Memory&#8221; tree gives the high-level overview.\u00a0 The root of that tree is the total memory used by the process (which the old about:memory did not list).<\/li>\n<li>The &#8220;Used Heap Memory&#8221; tree gives the breakdown for the used part of the heap (which corresponds to the &#8220;mapped\/heap\/used&#8221; figure in the &#8220;Mapped Memory&#8221; tree).\u00a0 I show the used heap separately because most of the time that&#8217;s the most interesting information.\u00a0 In contrast, the &#8220;Mapped Memory&#8221; tree is often dominated by code and data segments, which don&#8217;t correspond to explicit memory allocation requests from the process and so are less variable and harder to improve.<\/li>\n<li>&#8220;Other Measurements&#8221; holds measurements that overlap with the tree  breakdown.\u00a0 The new &#8220;resident&#8221; figure tells you how much physical memory  is being used by the process.<\/li>\n<li>Entries that cover a very small section of memory (less than 0.1% of the used heap, for the &#8220;Used Heap Memory&#8221; tree) are aggregated into entries that look like &#8220;(2 omitted)&#8221;.\u00a0 This means that more memory reporters can be added in the future without the output being cluttered up by tiny entries.<\/li>\n<li>If you click on the &#8220;More verbose&#8221; link at the bottom, the page redraws showing all the sizes as bytes instead of MB, and aggregated entries are shown in full.<\/li>\n<li>It handles multiple processes.\u00a0 If you view about:memory in Fennec you get output for each process separately.\u00a0 The plug-in container process isn&#8217;t represented at the moment, though;\u00a0 <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=648415\">bug 648415<\/a> is open for adding that, though it&#8217;s of less interest since plug-ins like Flash are out of Mozilla&#8217;s control.<\/li>\n<li>You can&#8217;t see it in the picture, but the tool-tips describing each metric are clearer, more consistently expressed, and expressed as complete sentences.<\/li>\n<li>The formatting is much more similar to other &#8220;about:&#8221; pages such as <a href=\"about:cache\">about:cache<\/a> and <a href=\"about:buildconfig\">about:buildconfig<\/a>.<\/li>\n<li>You can cut and paste the output into a text box and it reproduces beautifully.\u00a0 The lines used to represent the tree structure are <a href=\"http:\/\/en.wikipedia.org\/wiki\/Box-drawing_characters\">Unicode box-drawing characters<\/a>, and I stuck to the subset that are most commonly supported by terminals.\u00a0 The generated HTML has newlines in just the right places so that whitespace is inserted nicely.\u00a0 Here&#8217;s an example:<\/li>\n<\/ul>\n<pre>Main Process\r\n\r\nMapped Memory\r\n634.89 MB (100.0%) -- mapped\r\n\u251c\u2500\u2500526.33 MB (82.90%) -- other\r\n\u251c\u2500\u2500106.00 MB (16.70%) -- heap\r\n\u2502  \u251c\u2500\u2500\u250053.34 MB (08.40%) -- used\r\n\u2502  \u2514\u2500\u2500\u250052.66 MB (08.29%) -- unused\r\n\u2514\u2500\u2500\u2500\u25002.56 MB (00.40%) -- js\r\n     \u251c\u2500\u25002.44 MB (00.38%) -- mjit-code\r\n     \u2514\u2500\u25000.13 MB (00.02%) -- tjit-code\r\n\r\nUsed Heap Memory\r\n53.34 MB (100.0%) -- heap-used\r\n\u251c\u2500\u250022.22 MB (41.65%) -- js\r\n\u2502  \u251c\u2500\u250021.00 MB (39.37%) -- gc-heap\r\n\u2502  \u251c\u2500\u2500\u25000.75 MB (01.41%) -- tjit-data\r\n\u2502  \u2502   \u2514\u2500\u25000.75 MB (01.41%) -- allocators\r\n\u2502  \u2502      \u251c\u2500\u25000.56 MB (01.06%) -- reserve\r\n\u2502  \u2502      \u2514\u2500\u25000.19 MB (00.35%) -- main\r\n\u2502  \u251c\u2500\u2500\u25000.35 MB (00.66%) -- mjit-data\r\n\u2502  \u2514\u2500\u2500\u25000.11 MB (00.21%) -- string-data\r\n<\/pre>\n<p>The screenshots above are from my Ubuntu Linux box.\u00a0 Funnily enough, on Mac the lines in the box-drawing characters don&#8217;t line up consistently so the output doesn&#8217;t look as nice, as this screenshot shows:<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-mac.png\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-675 aligncenter\" title=\"aboutmemory-mac\" src=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-mac.png\" alt=\"New about:memory output on Mac, showing misaligned box characters\" width=\"579\" height=\"317\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-mac.png 579w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/05\/aboutmemory-mac-300x164.png 300w\" sizes=\"(max-width: 579px) 100vw, 579px\" \/><\/a><\/p>\n<p>Not much that can be done about that, as far as I know;\u00a0 it&#8217;s certainly not a show-stopper.<\/p>\n<p>My hope is that this revamped about:memory will be the first place both users and developers look when trying to understand any issue related to memory usage.\u00a0 For example, it should subsume OS memory reporters like &#8216;top&#8217;, &#8216;ps&#8217; and the Windows task manager.\u00a0 This will make bug reports easier to understand;\u00a0 in particular it will help avoid the problem where someone says &#8220;Firefox is using 800MB of memory&#8221; and you don&#8217;t know which metric they are using.<\/p>\n<p>There are some definite improvements to be made.\u00a0 Most notably, the &#8220;heap-used\/other&#8221; entry is usually the largest one in the &#8220;Heap Memory Used&#8221; tree &#8212; large chunks of memory aren&#8217;t being covered by memory reporters, so new ones need to be added.\u00a0 Working out what these reporters are won&#8217;t be easy, but Massif should be a big help.<\/p>\n<p>After that, it&#8217;d be great to have reporting at the level of individual tabs or something like that.\u00a0 Mike Shaver has a <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=625305\">draft patch<\/a> adding an about:compartments page which reports about JavaScript memory usage for individual compartments, which correspond to domains.\u00a0 This is a great start, though I&#8217;d like non-JavaScript memory usage to also be divided in this way.\u00a0 The details of how best to do this are not yet clear to me.<\/p>\n<p>The new about:memory is on track to make it into Firefox 6.\u00a0 Hopefully it will help reduce Firefox&#8217;s memory consumption.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A while ago I wrote about some ideas I had for improving about:memory.\u00a0 Bug 653633 has been tracking the first stage of this (and it now also has a feature page) and it only needs to pass super-review to be ready for landing. Here&#8217;s what the old about:memory looks like. Some things to note: The [&hellip;]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4550,30,4545,4544],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/671"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=671"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/671\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=671"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=671"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=671"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}