{"id":522,"date":"2011-02-09T04:48:05","date_gmt":"2011-02-08T17:48:05","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=522"},"modified":"2011-05-12T14:09:35","modified_gmt":"2011-05-12T03:09:35","slug":"a-vision-for-better-memory-profiling-with-aboutmemory","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2011\/02\/09\/a-vision-for-better-memory-profiling-with-aboutmemory\/","title":{"rendered":"A vision for better memory profiling with about:memory"},"content":{"rendered":"<p>I&#8217;ve been doing lots of memory profiling of Firefox.\u00a0 I&#8217;ve mostly used Massif and <a href=\"about:memory\">about:memory<\/a> for this.\u00a0 Both have their uses, but also big problems.\u00a0 I have some ideas for combining their best features into a killer profiling tool.<\/p>\n<h3>Massif&#8217;s pros and cons<\/h3>\n<p>I&#8217;ve mostly used Massif.\u00a0 It&#8217;s both invaluable and a total pain.\u00a0 More specifically, it has the following advantages:<\/p>\n<ul>\n<li>Time series.\u00a0 It shows how memory consumption changes over time, both graphically (for total allocations) and with periodic detailed snapshots.<\/li>\n<li>Level of detail.\u00a0 Each detailed snapshot consists of an allocation tree that shows which parts of the code are responsible for every single byte allocated.<\/li>\n<\/ul>\n<p>But it has lots of disadvantages.<\/p>\n<ul>\n<li>Slow.\u00a0 There&#8217;s somewhere between a 10x and 100x slowdown;\u00a0 that&#8217;s a rough estimate, I haven&#8217;t actually measured it.<\/li>\n<li>It&#8217;s implemented with Valgrind, so it doesn&#8217;t work on Windows.<\/li>\n<li>It&#8217;s not easy to use.\u00a0 The command-line needed to get good results with Firefox is <a href=\"http:\/\/blog.mozilla.org\/nnethercote\/2011\/01\/07\/memory-profiling-firefox-with-massif-part-2\/\">huge<\/a>.<\/li>\n<li>There&#8217;s little control over when snapshots occur.\u00a0 That could be improved relatively easily with client requests, though they require code modifications so they&#8217;re a bit painful.<\/li>\n<li>The superblock allocation problem.\u00a0 You can profile with Massif at the OS page level, or at the heap (malloc) level.\u00a0 In both cases, the results can be misleading because an allocation request doesn&#8217;t always result in a visible allocation occurring.\u00a0 For example, if profiling at the OS page level, most calls to malloc won&#8217;t result in pages being allocated with mmap, because when jemalloc needs pages from the OS it will usually request more than necessary for the current request, and then hand out pieces of those pages itself on subsequent calls to malloc.\u00a0 If profiling at the heap level, this problem also occurs with custom allocators such as JSArenaPool that are layered on top of malloc.\u00a0 As a result, many allocation requests don&#8217;t get recorded by Massif, and a small number of them are blamed for allocating much more memory than they actually did.\u00a0 (The whole question of &#8220;what level do I measure at?&#8221; is one of the trickiest things about memory profiling.)<\/li>\n<\/ul>\n<p>The result is that using Massif and understanding its output is difficult for anyone who isn&#8217;t an expert.<\/p>\n<h3>about:memory&#8217;s pros and cons<\/h3>\n<p>The other tool I sometimes use is about:memory.\u00a0 It has the following advantages:<\/p>\n<ul>\n<li>Lightweight, trivial to view.<\/li>\n<li>Full control over when measurements occur.<\/li>\n<\/ul>\n<p>And the disadvantages:<\/p>\n<ul>\n<li>Minimal data, mostly just the heap.<\/li>\n<li>The values are not as easy to understand as they seem.<\/li>\n<li>No time series.<\/li>\n<\/ul>\n<h3>A better about:memory<\/h3>\n<p>I want the best of both worlds:\u00a0 time series, detailed measurements, control when measurements occur, ease-of-use.\u00a0 Plus a bit more: both OS page and heap level measurements, accurate numbers, global and per-compartment measurements, and good visualizations of both snapshots and time series data.\u00a0 Basically, I want to re-implement a decent chunk of Massif within about:memory.\u00a0 What follows is a 4am braindump, apologies for the density, hopefully it&#8217;s mostly understandable.<\/p>\n<p>Merge about:memory and Shaver&#8217;s nascent <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=625305\">about:compartments<\/a>, because they&#8217;re really doing the same thing.<\/p>\n<p>More global measurements:<\/p>\n<ul>\n<li>Add current and peak total memory and RSS (resident set size), as measured by the system (eg. \/proc\/pid\/status on Linux).<\/li>\n<li>Keep malloc stats as they currently are (eg. malloc\/allocated).<\/li>\n<li>More individual counts.\u00a0 Eg. for JS we currently have js\/gc-heap, js\/string-data, js\/mjit-code.\u00a0 I want to split js\/mjit-code into several counts: JaegerMonkey code (inline, out-of-line, ICs) and JaegerMonkey data (JITScripts and IC data).\u00a0 And also add counts for: TraceMonkey code and data, Shapes, JSFunctions, JSScripts, cx-&gt;tempAlloc (which holds JSParseNodes plus other things), JSFunctions, js::Vector, js::HashTable&#8230; basically anything that shows up reasonably high in Massif profiles.\u00a0 We currently have 19 individual counts, I imagine having 50+.\u00a0 Obviously there&#8217;ll be lots of non-JS counters added as well.<\/li>\n<li>Allow individual counts to be shown in a standard order, or in order from biggest to smallest.<\/li>\n<li>For a lot of these, both the number of bytes allocated and the number of allocations might be useful.<\/li>\n<\/ul>\n<p>Per-compartment measurements: show all the individual counts, but on a per-compartment basis.<\/p>\n<p>Clearer meanings of measurements:<\/p>\n<ul>\n<li>Add links\/tooltips\/whatever to every measurement with a brief description so that it&#8217;s clear what it means.<\/li>\n<li>For individual counts, clearly distinguish heap allocations from non-heap allocations (eg. executable code space allocated by the JITs).<\/li>\n<li>Have &#8220;everything else&#8221; measurements for both the heap and the total, found by subtracting individual counts from the overall and heap totals.<\/li>\n<\/ul>\n<p>Visualization of measurements.\u00a0 E.g.:<\/p>\n<ul>\n<li> Show the heap as proportion of total memory.<\/li>\n<li>Show each individual global count as a proportion of the total.<\/li>\n<li>Show each compartment&#8217;s individual count sum as a proportion of the global individual count sum.<\/li>\n<li>All this via pie charts or similar.<\/li>\n<\/ul>\n<p>Time series:<\/p>\n<ul>\n<li>Allow about:memory snapshots to be dumped to file in some manner, probably as JSON.<\/li>\n<li>Do it in a way that allows multiple snapshots to be processed easily by an external tool.<\/li>\n<li>Basically, Sayre&#8217;s <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=631705\">membuster<\/a> on steroids.<\/li>\n<li>Furthermore, make that data available to the browser itself as well.<\/li>\n<\/ul>\n<p>Visualization of time series data:<\/p>\n<ul>\n<li>Use the time series data to show a graph in about:memory.<\/li>\n<li>Make the graph interactive, eg. allow drilling down to individual counts, going back to previous snapshots.<\/li>\n<li>I have a hacky prototype canvas implementation of something like this that reads Massif&#8217;s output files.\u00a0 SVG would probably be better, though.<\/li>\n<\/ul>\n<p>Diffs of some kind between snapshots would be great.\u00a0 It would allow you to answer questions like &#8220;how much memory is allocated when I open a new tab?&#8221;<\/p>\n<p>If <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=585196\">telemetry<\/a> is ever implemented, sending this data back from users would be great, though that&#8217;s a lot harder.<\/p>\n<p>Potential difficulties:<\/p>\n<ul>\n<li>Not that many, as far as I can tell.\u00a0 A lot of the infrastructure is already in place.<\/li>\n<li>Keeping the counter code up-to-date may be tricky.\u00a0 If it&#8217;s used frequently by many people, that&#8217;ll increase the likelihood that it&#8217;ll be kept up to date.\u00a0 Better descriptions will help make it clearer if the counters are counting what they&#8217;re supposed to.<\/li>\n<li>about:memory will itself use some memory.\u00a0 It&#8217;s unclear how to avoid measuring that.\u00a0 Maybe putting the more advanced features like the graphical stuff in a separate about:morememory page might mitigate this;\u00a0 you could take a bunch of snapshots via about:memory and then open up about:morememory.<\/li>\n<li>Performance&#8230; will more counters be noticeable?\u00a0 Hopefully not since we already have a bunch of them anyway.<\/li>\n<li>Unsophisticated users might file unhelpful &#8220;hey, Firefox is using too much memory&#8221;.\u00a0 But sophisticated users might file helpful &#8220;hey, Firefox is using too much memory&#8221; bugs.<\/li>\n<\/ul>\n<p>Basically, if this can be made as attractive and useful in reality as it currently is in my imagination, I figure no-one will ever need to use an external memory profiler for Firefox again.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I&#8217;ve been doing lots of memory profiling of Firefox.\u00a0 I&#8217;ve mostly used Massif and about:memory for this.\u00a0 Both have their uses, but also big problems.\u00a0 I have some ideas for combining their best features into a killer profiling tool. Massif&#8217;s pros and cons I&#8217;ve mostly used Massif.\u00a0 It&#8217;s both invaluable and a total pain.\u00a0 More [&hellip;]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4550,30,4545,4544],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/522"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=522"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/522\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=522"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=522"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=522"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}