{"id":1311,"date":"2011-09-07T17:25:20","date_gmt":"2011-09-07T06:25:20","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=1311"},"modified":"2011-09-07T17:25:20","modified_gmt":"2011-09-07T06:25:20","slug":"memshrink-progress-week-12","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2011\/09\/07\/memshrink-progress-week-12\/","title":{"rendered":"MemShrink progress, week 12"},"content":{"rendered":"<h3>about:memory improvements<\/h3>\n<p>Lots of changes were made to about:memory this week.<\/p>\n<p>Justin Lebar landed <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=674290\">a patch that provides detailed information about RSS, vsize and swap memory usage on Linux and Android<\/a>.\u00a0 (The patch was backed out due to a minor leak but I expect Justin will fix that and re-land it soon.)\u00a0 <a><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-1312\" title=\"rss\" src=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/rss.png\" alt=\"\" width=\"630\" height=\"392\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/rss.png 630w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/rss-300x186.png 300w\" sizes=\"(max-width: 630px) 100vw, 630px\" \/><\/a> This will help us understand memory usage that is not covered by the &#8220;Explicit Allocations&#8221; tree in about:memory, such as memory used for static code and data, and it should be particularly useful on Android.\u00a0 The contents of the new trees are hidden by default;\u00a0 you have to click on the tree heading to expand each one.<\/p>\n<p>Kyle Huey <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=683149\">split up about:memory&#8217;s layout measurements on a per-PresShell basis<\/a>.\u00a0 This makes it easy to see how much layout memory is being used by each web page.<\/p>\n<p><a href=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/layout-shell.png\"><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-1314\" title=\"layout-shell\" src=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/layout-shell.png\" alt=\"\" width=\"642\" height=\"399\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/layout-shell.png 642w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/layout-shell-300x186.png 300w\" sizes=\"(max-width: 642px) 100vw, 642px\" \/><\/a><\/p>\n<p>Something I failed to mention <a href=\"http:\/\/blog.mozilla.org\/nnethercote\/2011\/08\/31\/memshrink-progress-week-11\/\">last week <\/a>was that with the landing of type inference, each JavaScript compartment has five new measurements: &#8220;object-main&#8221;, &#8220;script-main&#8221;, &#8220;tables&#8221;, &#8220;analysis-temporary&#8221;, and &#8220;object-empty-shapes&#8221;.<\/p>\n<p><a href=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/ti2.png\"><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-1316\" title=\"ti2\" src=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/ti2.png\" alt=\"\" width=\"495\" height=\"130\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/ti2.png 495w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/09\/ti2-300x78.png 300w\" sizes=\"(max-width: 495px) 100vw, 495px\" \/><\/a><\/p>\n<p>I converted some of the JavaScript memory reporters to use <code>moz_malloc_usable_size<\/code> to measure actual allocation sizes instead of requested allocation sizes.\u00a0 This accounts for slop bytes caused by the heap allocator rounding up.\u00a0 This is quite important &#8212; slop bytes can easily account for over 10% of the heap, and if we don&#8217;t account for them we&#8217;ll never get about:memory&#8217;s &#8220;heap-unclassified&#8221; number down.\u00a0 Therefore I&#8217;ll be doing <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=684799\">more<\/a> of <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=684800\">this<\/a> in the future.\u00a0 And it would be great if people writing new memory reporters can do the same thing!<\/p>\n<p>Finally, on the topic of &#8220;heap-unclassified&#8221; number:\u00a0 people often complain about it, so I&#8217;m happy to say that it is on a clear downward path.\u00a0 Indeed, thanks to <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=676724\">DMD<\/a>, at the time of writing we have <a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?status_whiteboard_type=allwordssubstr&amp;query_format=advanced&amp;list_id=1252756&amp;short_desc=memory%20reporter&amp;status_whiteboard=MemShrink&amp;bug_status=UNCONFIRMED&amp;bug_status=NEW&amp;bug_status=ASSIGNED&amp;bug_status=REOPENED&amp;short_desc_type=regexp&amp;resolution=---&amp;resolution=DUPLICATE\">16 open bugs to add new memory reporters<\/a> for things that consume significant amounts of memory, and 13 of these are assigned.\u00a0 I&#8217;m hoping that in a month or two the &#8220;heap-unclassified&#8221; number on development builds will typically be 10&#8211;15% rather than the 30&#8211;35% it usually is now.<\/p>\n<h3>Other things<\/h3>\n<p>I <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=684180\">changed the growth strategy used for one of JaegerMonkey&#8217;s buffers<\/a> to avoid large amounts of memory wasted due to slop bytes.\u00a0 These buffers are short-lived so the fix doesn&#8217;t make a great difference to total memory consumption, but it does reduce the number of allocations and heap churn.<\/p>\n<p>Marco Bonardo wrote about <a href=\"http:\/\/blog.bonardo.net\/2011\/09\/02\/changes-to-places-memory-consumption-expiration-and-performances\">his recent changes to the handling of the places database<\/a>.<\/p>\n<p>Dietrich Ayala wrote about <a href=\"http:\/\/autonome.wordpress.com\/2011\/09\/02\/dormancy-freeing-up-memory-from-unused-tabs\/\">an experimental add-on that unloads tabs that haven&#8217;t been viewed in a while<\/a>, which is an interesting idea.\u00a0 I suspect the exact approach used in the add-on won&#8217;t be viable in the long run, but we can certainly benefit from doing a better job of <a href=\"http:\/\/groups.google.com\/group\/mozilla.dev.platform\/browse_thread\/thread\/44a0efec8a336867?pli=1\">discarding regenerable info that hasn&#8217;t been used in a while<\/a>, particularly on mobile.<\/p>\n<h3>Bug counts<\/h3>\n<p>This weeks&#8217;s bug counts are as follows:<\/p>\n<ul>\n<li>P1: 29 (-2, +2)<\/li>\n<li>P2: 80 (-4, +8)<\/li>\n<li>P3: 40 (-2, +4)<\/li>\n<li>Unprioritized: 22 (-12, +12)<\/li>\n<\/ul>\n<p>Just like last week, Marco Castelluccio tagged quite a lot of old bugs with &#8220;[MemShrink]&#8221;.\u00a0 We had 45 unprioritized bugs at the start of this week&#8217;s meeting, and we got through more than 20 of them.<\/p>\n<p>Some comments on last week&#8217;s post got me thinking about how to make it easier for more people to help with MemShrink.\u00a0 For those who don&#8217;t have much coding experience, probably the best bet is to look at the <a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?status_whiteboard_type=allwordssubstr&amp;query_format=advanced&amp;list_id=1252875&amp;status_whiteboard=MemShrink&amp;bug_status=UNCONFIRMED&amp;resolution=---&amp;resolution=DUPLICATE\">list of unconfirmed bugs<\/a> &#8212; these are problems reported by users where the particular problem hasn&#8217;t been identified.\u00a0 Often they need additional effort to determine if they are reproducible, due to add-ons, etc.\u00a0 For example, in <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=676872\">bug 676872<\/a> a user was seeing very high memory usage, and it&#8217;s clear that it was caused by one or more of the 41(!) add-ons he had enabled.\u00a0 Ideally that bug&#8217;s reporter would disable them selectively to narrow that down, but anyone could do likewise with some effort.<\/p>\n<p>For those who do have coding experience, it would be worth looking at the <a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?status_whiteboard_type=allwordssubstr&amp;query_format=advanced&amp;list_id=1252905&amp;status_whiteboard=MemShrink%20mentor&amp;bug_status=UNCONFIRMED&amp;bug_status=NEW&amp;bug_status=ASSIGNED&amp;bug_status=REOPENED&amp;resolution=---&amp;resolution=DUPLICATE\">list of bugs that have a &#8220;mentor&#8221; annotation<\/a>.\u00a0 For example, <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=472209\">bug 472209<\/a> is about adding some graphing capability to about:memory.\u00a0 Jezreel Ng made some excellent progress on this during his internship, it just needs someone to take over and finish it up.<\/p>\n<p>Finally, for those who like a challenge or have some experience with Firefox&#8217;s code, the <a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?list_id=1252950&amp;resolution=---&amp;resolution=DUPLICATE&amp;emailtype1=regexp&amp;status_whiteboard_type=allwordssubstr&amp;emailassigned_to1=1&amp;query_format=advanced&amp;status_whiteboard=MemShrink&amp;bug_status=UNCONFIRMED&amp;bug_status=NEW&amp;bug_status=ASSIGNED&amp;bug_status=REOPENED&amp;email1=nobody%40%7Cgeneral%40js.bugs\">full list of unassigned bugs<\/a> might be of interest.\u00a0 There are currently 86 such bugs!\u00a0 More than I&#8217;d like.<\/p>\n<p>(BTW, I&#8217;ve added links for the three bug lists above to the <a href=\"https:\/\/wiki.mozilla.org\/Performance\/MemShrink#Bug_Tracking\">MemShrink wiki page<\/a>.)<a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?status_whiteboard_type=allwordssubstr&amp;query_format=advanced&amp;list_id=1252875&amp;status_whiteboard=MemShrink&amp;bug_status=UNCONFIRMED&amp;resolution=---&amp;resolution=DUPLICATE\"><br \/>\n<\/a><\/p>\n<h3>On HIATUS<\/h3>\n<p>I will be <a title=\"On vacation September 12 to October 17\" href=\"http:\/\/blog.mozilla.org\/nnethercote\/2011\/09\/07\/on-vacation-september-12-to-october-17\/\">on vacation for the next five weeks<\/a> and the MemShrink progress report will be on hiatus during that time.\u00a0 But MemShrink meetings will continue (except there won&#8217;t be one next week due to the Mozilla all-hands meeting).\u00a0 I look forward to writing a bumper progress report on October 19, where I&#8217;ll be able to summarize everything that happened while I was away!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>about:memory improvements Lots of changes were made to about:memory this week. Justin Lebar landed a patch that provides detailed information about RSS, vsize and swap memory usage on Linux and Android.\u00a0 (The patch was backed out due to a minor leak but I expect Justin will fix that and re-land it soon.)\u00a0 This will help [&hellip;]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4550,30,4544,4546],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/1311"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=1311"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/1311\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=1311"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=1311"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=1311"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}