{"id":1447,"date":"2011-11-09T15:11:55","date_gmt":"2011-11-09T04:11:55","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=1447"},"modified":"2011-11-09T15:11:55","modified_gmt":"2011-11-09T04:11:55","slug":"memshrink-progress-week-21","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2011\/11\/09\/memshrink-progress-week-21\/","title":{"rendered":"MemShrink progress, week 21"},"content":{"rendered":"<h3>MemShrink:P1 Bugs fixed<\/h3>\n<p>Terrence Cole made a change that<a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=670596\"> allows unused arenas in the JS heap to be decommitted<\/a>, which means they more or less don&#8217;t cost anything.\u00a0 This helps reduce the cost of JS heap fragmentation, which is a good short-term step while we are waiting for a compacting garbage collector to be implemented.\u00a0 Terrence followed it up by <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=700357\">making the JS garbage collector do more aggressive collections when many decommitted arenas are present<\/a>.<\/p>\n<p>Justin Lebar <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=694896\">enabled jemalloc on MacOS 10.7<\/a>.\u00a0 This means that jemalloc is finally used on all supported versions of our major OSes: Windows, Mac, Linux and Android.\u00a0 Having a common heap allocator across these platforms is great for consistency of testing and behaviour, and makes future improvements involving jemalloc easier.<\/p>\n<p>Gabor Krizsanits\u00a0<a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=677294\">created a new API in the add-on SDK that allows multiple sandboxes to be put into the same JS compartment<\/a>.<\/p>\n<h3>Other Bugs Fixed<\/h3>\n<p>I <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=678977\">registered jemalloc with SQLite&#8217;s pluggable allocator interface<\/a>.\u00a0 This had two benefits.\u00a0 First, it means that SQLite no longer needs to store the size of each allocation next to the allocation itself, avoiding some <a href=\"http:\/\/blog.mozilla.org\/nnethercote\/2011\/08\/05\/clownshoes-available-in-sizes-2101-and-up\/\">clownshoes<\/a> allocations that wasted space.\u00a0 This reduces SQLite&#8217;s total memory usage by a few percent.\u00a0 Second, it makes the SQLite numbers in about:memory 100% accurate;\u00a0 previously SQLite was under-reporting its memory usage, sometimes significantly.<\/p>\n<p>Relatedly, Marco Bonardo made three changes (<a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=695554\">here<\/a>, <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=658303\">here<\/a> and <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=700417\">here<\/a>) that reduce the amount of memory used by the Places database.<\/p>\n<p>Peter Van der Beken <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=699796\">fixed a cycle collector leak<\/a>.<\/p>\n<p>I <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=684800\">tweaked the JavaScript type inference memory reporters<\/a> to provide better coverage.<\/p>\n<p>Jiten <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=683517\">increased the amount of stuff that is released on memory pressure events<\/a>, which are triggered when Firefox on Android moves to the background.<\/p>\n<p>Finally, I created a meta-bug for tracking <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=700547\">add-ons that are known to have memory leaks<\/a>.<\/p>\n<h3>Bug Counts<\/h3>\n<p>I accidentally deleted my record of the live bugs from last week, so I don&#8217;t have the +\/- numbers for each priority this week.<\/p>\n<ul>\n<li>P1: 29 (last week: 35)<\/li>\n<li>P2: 126 (last week: 116)<\/li>\n<li>P3: 59 (last week: 55)<\/li>\n<li>Unprioritized: 0 (last week: 5)<\/li>\n<\/ul>\n<p>The P1 result was great this week &#8212; six fewer than last week.\u00a0 Three of those were fixed, and three of those I downgraded to P2 because they&#8217;d been partially\u00a0 addressed.<\/p>\n<p>For a longer view of things, here is a graph showing the MemShrink bug count since the project started in early June.<\/p>\n<p><a href=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/11\/memshrink.png\"><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-1453\" title=\"memshrink\" src=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/11\/memshrink.png\" alt=\"memshrink bug count\" width=\"710\" height=\"250\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/11\/memshrink.png 710w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2011\/11\/memshrink-300x105.png 300w\" sizes=\"(max-width: 710px) 100vw, 710px\" \/><\/a><\/p>\n<p>There was an early spike as many existing bugs were tagged with &#8220;MemShrink&#8221;, and a smaller spike in the middle when Marco Castellucio tagged a big pile of older bugs.\u00a0 Other than that, the count has marched steadily upward at the rate of about six per week.\u00a0 Many bugs are being fixed and definite improvements are being made, but this upward trend has been concerning me.<\/p>\n<h3>Future Directions<\/h3>\n<p>So in today&#8217;s MemShrink meeting we spent some time discussing future directions of MemShrink.\u00a0 Should we continue as is?\u00a0 Should we change our focus, e.g. by concentrating more on mobile, or setting some specific targets?<\/p>\n<p>The discussion was long and wide-ranging and not easy to summarize.\u00a0 One topic was &#8220;what is the purpose of MemShrink?&#8221;\u00a0 The point being that memory usage is really a secondary measure.\u00a0 By and large, people don&#8217;t really care how many MB of memory Firefox is using;\u00a0 they care how responsive it is, and it&#8217;s just assumed that reducing memory usage will help with that.\u00a0 With that in mind, I&#8217;ll attempt to paraphrase and extrapolate some goals (apologies if I&#8217;ve misrepresented people&#8217;s opinions).<\/p>\n<ul>\n<li>On 64-bit desktop, the primary goal is that Firefox&#8217;s performance should not degrade after using it heavily (e.g. many tabs) for a long time.\u00a0 This means it shouldn&#8217;t page excessively, and that operations like garbage collection and cycle collection shouldn&#8217;t get slower and slower.<\/li>\n<li>On mobile, the primary goal probably is to reduce actual memory usage.\u00a0 This is because usage on mobile tends to be lighter (e.g. not many tabs) so the longer term issues are less important.\u00a0 However, Firefox will typically be killed by the OS if it takes up too much memory.<\/li>\n<li>On 32-bit desktop, both goals are relevant.<\/li>\n<\/ul>\n<p>As for how these goals would change our process, it&#8217;s not entirely clear.\u00a0 For desktop, it would be great to have a benchmark that simulates a lot of browsing (opening and closing many sites and interacting with them in non-trivial ways).\u00a0 At the end we could measure various things, such a memory usage, garbage and cycle collection time, and we could set targets to reduce those.\u00a0 For mobile, the current MemShrink process probably doesn&#8217;t need to change that much, though more profiling on mobile devices would be good.<\/p>\n<p>Personally, I&#8217;ve been spreading myself thinly over a lot of MemShrink bugs.\u00a0 In particular, I try to push them along and not let them stall by doing things like trying to reproduce them, asking questions, etc.\u00a0 I&#8217;ve been feeling lately like it would be a better use of my time to do less of that and instead dig deeply into a particular area.\u00a0 I thought about working on <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=678037\">making JS script compilation lazy<\/a>, but now I&#8217;ve decided instead to focus primarily on improving the measurements in about:memory, in particular, reducing the size of &#8220;heap-unclassified&#8221; by improving existing memory reporters and adding new ones. I&#8217;ve decided this because it&#8217;s an area where I have expertise, clear ideas on how to make progress, and <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=676724\">tools<\/a> to help me.\u00a0 Plus it&#8217;s important;\u00a0 we can&#8217;t make improvements without measurements, and about:memory is the best memory measurement tool we have.\u00a0 Hopefully other people agree that this is important to work on \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"<p>MemShrink:P1 Bugs fixed Terrence Cole made a change that allows unused arenas in the JS heap to be decommitted, which means they more or less don&#8217;t cost anything.\u00a0 This helps reduce the cost of JS heap fragmentation, which is a good short-term step while we are waiting for a compacting garbage collector to be implemented.\u00a0 [&hellip;]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4550,30,4555,4544,4546,4551],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/1447"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=1447"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/1447\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=1447"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=1447"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=1447"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}