{"id":1227,"date":"2011-08-17T16:54:01","date_gmt":"2011-08-17T05:54:01","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=1227"},"modified":"2011-08-20T09:10:14","modified_gmt":"2011-08-19T22:10:14","slug":"memshrink-progress-week-9","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2011\/08\/17\/memshrink-progress-week-9\/","title":{"rendered":"MemShrink progress, week 9"},"content":{"rendered":"<p>Firefox 8 graduated to the <a href=\"http:\/\/aurora.mozilla.org\/\">Aurora channel<\/a> this week, and the development period for what will become Firefox 9 began.\u00a0 Lots of MemShrink activity happened this week, and I think all the changes listed below will make it into Firefox 8.<\/p>\n<h3>Avoiding Wasted Memory<\/h3>\n<p>I have blogged previously about memory wasted by <a href=\"http:\/\/blog.mozilla.org\/nnethercote\/2011\/08\/05\/clownshoes-available-in-sizes-2101-and-up\/\">&#8220;clownshoes&#8221; bugs<\/a>. \u00a0 Ed Morley found a webpage that resulted in <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=678376\">700MB of memory being wasted<\/a> by the <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=676457\">PLArena clownshoes bug<\/a>.\u00a0 Basically, on platforms where jemalloc is used (Windows, Linux), half the memory allocated by nsPresArena (which is built on top of PLArena) was wasted.\u00a0 (On Mac the waste was 11%, because the Mac allocator rounds up less aggressively than jemalloc).<\/p>\n<p>Fixing this problem properly for all PLArenas takes time because it requires changes to NSPR, so I made a <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=678422\">spot-fix<\/a> for the nsPresArena case.\u00a0 This is a particularly big win on very large pages, but it saves around 3MB even on Gmail. This spot-fix has been granted beta approval and so will, barring disaster, make it into Firefox 7.<\/p>\n<p>A Firefox Nightly user did some <a href=\"http:\/\/www.neowin.net\/forum\/topic\/989780-meet-firefox-next\/page__st__720__p__594231040#entry594231040\">measurements<\/a> with different browsers on the problematic page:<\/p>\n<ul>\n<li>Firefox 8.0a1 before patch: 2.0 GB<\/li>\n<li>Firefox 8.0a1 after patch: 1.3 GB<\/li>\n<li>Latest Chrome canary build and dev (15.0.849.0): 1.1GB<\/li>\n<li>Webkit2Process of Safari 5.1: 1.05 GB<\/li>\n<li>Internet Explorer 9.0.2: 838 MB<\/li>\n<li>Latest Opera Next 12.00: 727 MB<\/li>\n<\/ul>\n<p>So this fix gets Firefox within spitting distance of other browsers, which is good!<\/p>\n<p>In other developments related to avoiding wasted memory:<\/p>\n<ul>\n<li>Luke Wagner discovered that, on typical websites, <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=678037\">most JSScripts are byte-compiled but never run<\/a>.\u00a0 A JSScript roughly corresponds to a JavaScript function.\u00a0 In hindsight, it&#8217;s not such a surprising result &#8212; Firefox byte-compiles all loaded JavaScript code, and you can imagine lots of websites use libraries like jQuery but only use a small fraction of the functions in the library.\u00a0 Making byte-compilation lazy could potentially save MBs of memory per compartment.\u00a0 But that will require some non-trivial reworking of the JS engine, and so is unlikely to happen in the short-term.<\/li>\n<li>Kyle Huey <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=679191\">avoided a small amount (~100KB per browser process) of waste due to rounding up in XPT arenas<\/a>.<\/li>\n<\/ul>\n<h3>Improving about:memory<\/h3>\n<p>I made some progress on a Valgrind tool to help identify the memory that is currently reported only by the &#8220;heap-unclassified&#8221; bucket in about:memory.\u00a0 It&#8217;s called <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=676724\">&#8220;DMD&#8221;<\/a>, short for &#8220;Dark Matter Detector&#8221;.\u00a0 It&#8217;s in early stages and I still need to teach it about most of Firefox&#8217;s memory reporters, but it&#8217;s already spitting out useful data, which led to me and Ehsan Akhgari landing memory reporters for the <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=677466\">JS atom table<\/a> and the <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=669116\">Hunspell spell checker<\/a>.\u00a0 We also now have some insight (<a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=678376#c10\">here<\/a> and <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=678376#c13\">here<\/a>) about memory usage for very large pages.<\/p>\n<p>Mounir Lamouri <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=677506\">turned on the memory reporter for the DOM<\/a> that he&#8217;s been working on for some time.\u00a0 This shows up under &#8220;dom&#8221; in about:memory.\u00a0 There are still some cases that require handling;\u00a0 you can follow the progress of these <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=663271\">here<\/a>.<\/p>\n<p>Andrew McCreight replaced about:memory&#8217;s buttons so you can <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=677358\">force a cycle collection without also forcing a garbage collection<\/a>, which may be useful in hunting down certain problems.<\/p>\n<p>Finally, Sander van Veen added the existing <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=673837\">&#8220;js-compartments-user&#8221; and &#8220;js-compartments-system&#8221; to the statistics collected by telemetry<\/a> (his first landed patch!), and I <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=678922\">did likewise for the &#8220;storage\/sqlite&#8221;<\/a> reporter.\u00a0 I also added a new <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=678126\">&#8220;tjit-data\/trace-monitor&#8221; memory reporter<\/a> that accounts for some of the memory used by TraceMonkey.<\/p>\n<h3>Miscellaneous<\/h3>\n<p>Igor Bukanov <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=673795\">tweaked the handling of empty chunks by the JavaScript garbage collector<\/a>.\u00a0 That sounds boring until you see the <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=673795#c30\">results<\/a> on <a href=\"http:\/\/gregor-wagner.com\/tmp\/mem\">Gregor Wagner&#8217;s 150-tab stress test<\/a>: resident memory usage dropped 9.5% with all 150 tabs open, and dropped by 27% after all those tabs were closed.<\/p>\n<p>Brian Hackett <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=678029\">fixed a memory leak in type inference<\/a>, which gets it one step closer to being ready to land.<\/p>\n<p>Christian H\u00f6ltje <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=672619\">fixed a leak in his &#8220;It&#8217;s All Text&#8221; add-on<\/a> that was causing <a href=\"http:\/\/blog.mozilla.org\/nnethercote\/2011\/07\/20\/zombie-compartments-recognize-and-report-them-stop-the-screaming\/\">zombie compartments<\/a>.\u00a0 This fix will be in version 1.6.0, which is currently awaiting to receive AMO approval, but can be obtained <a href=\"https:\/\/addons.mozilla.org\/en-US\/firefox\/addon\/its-all-text\/versions\/\">here<\/a> in the meantime.\u00a0 This fix and last week&#8217;s fix of a memory leak in <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=674535\">LastPass<\/a> are very encouraging &#8212; per-compartment reporters in about:memory have, for the first time, given add-on developers a reasonable tool for identifying memory leaks.\u00a0 I hope we can continue to improve the situation here.\u00a0 Several people have asked me for documentation on how to avoid memory leaks in add-ons.\u00a0 I&#8217;m not the person to write that guide (I&#8217;m not a Gecko expert and I know almost nothing about add-ons) but hopefully someone else can step up to the plate.<\/p>\n<h3>Bug counts<\/h3>\n<p>Here\u2019s the change in MemShrink bug counts.<\/p>\n<ul>\n<li>P1: 30 (-0, +1)<\/li>\n<li>P2: 64 (-4, +6)<\/li>\n<li>P3: 36 (-5, +0)<\/li>\n<li>Unprioritized: 1 (-2, +1)<\/li>\n<\/ul>\n<p>Good progress on P3 bugs, but they&#8217;re the least important ones.\u00a0 Other than that, new bugs are still being reported faster than they&#8217;re being fixed.\u00a0 If you want to help but don&#8217;t know where to start, feel free to email me or ping me on IRC and I&#8217;ll do my best to help get you involved.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Firefox 8 graduated to the Aurora channel this week, and the development period for what will become Firefox 9 began.\u00a0 Lots of MemShrink activity happened this week, and I think all the changes listed below will make it into Firefox 8. Avoiding Wasted Memory I have blogged previously about memory wasted by &#8220;clownshoes&#8221; bugs. \u00a0 [&hellip;]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4550,30,4555,4556,4544,4546,467,484],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/1227"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=1227"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/1227\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=1227"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=1227"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=1227"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}