{"id":2020,"date":"2012-07-19T11:46:01","date_gmt":"2012-07-19T00:46:01","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=2020"},"modified":"2012-07-19T14:39:42","modified_gmt":"2012-07-19T03:39:42","slug":"firefox-15-plugs-the-add-on-leaks","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2012\/07\/19\/firefox-15-plugs-the-add-on-leaks\/","title":{"rendered":"Firefox 15 plugs the add-on leaks"},"content":{"rendered":"<h3>TL;DR<\/h3>\n<p>Firefox 15 prevents most memory leaks caused by add-ons, including Firebug.\u00a0 For many users with add-ons installed this will significantly reduce Firefox&#8217;s memory consumption, without requiring upgrades to those add-ons.<\/p>\n<p>For those users, Firefox 15 is likely to be faster (sometimes drastically so) and less likely to crash, especially if they have multiple add-ons installed and\/or keep Firefox running for a long time between restarts.<\/p>\n<h3>The Problem<\/h3>\n<p>A <a href=\"http:\/\/en.wikipedia.org\/wiki\/Memory_leak\"><em>memory leak<\/em><\/a> occurs when a program allocates some memory but then fails to free it.\u00a0 There are multiple causes of memory leaks, but they all have the same effect:\u00a0 an increase in memory consumption, which often grows as the program runs. This can cause poor performance, and even lead to crashes or aborts due to the exhaustion of available memory.<\/p>\n<h4>Many Add-ons Leak<\/h4>\n<p>I&#8217;ve lost count of the number of times I&#8217;ve seen (and participated in) variants of the following conversation during the past year.<\/p>\n<pre>Alice: Firefox is slow and using 2GB of memory.  WTF?\r\nBob:\u00a0  Do you have any add-ons installed?\r\nAlice: Yes.  Why?\r\nBob:\u00a0  Try disabling them all and restart Firefox.\r\nAlice: Ok... Oh wow.  That's much better.\r\nBob:\u00a0  Now you have to disable them one at a time to work\r\n       <em><\/em>out which one(s) are at fault.\r\nAlice: Really?  Hmm, that sucks.\r\nBob:\u00a0  Yeah.  Sorry about that.<\/pre>\n<p>Although Firefox itself isn&#8217;t immune to memory consumption problems, Mozilla&#8217;s <a href=\"https:\/\/wiki.mozilla.org\/Performance\/MemShrink\">MemShrink project<\/a> has <a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?list_id=3536501;status_whiteboard_type=allwordssubstr;status_whiteboard=MemShrink;resolution=FIXED;query_format=advanced;bug_status=RESOLVED;bug_status=VERIFIED;bug_status=CLOSED\">fixed enough problems<\/a> in Firefox itself that in January of this year I declared that the #1 way to improve memory consumption in Firefox would be with &#8220;<a href=\"http:\/\/blog.mozilla.org\/nnethercote\/2012\/01\/25\/memshrink-progress-week-32\/\">Better Detection and Notification of Leaky Add-ons<\/a>&#8220;.<\/p>\n<p>I said this because <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=700547\">many add-ons have memory leaks<\/a>.<\/p>\n<ul>\n<li>We have found leaks in popular, well-written add-ons, such as <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=672111\">Adblock Plus<\/a>, <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=731906\">Video DownloadHelper<\/a>, <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=707403\">GreaseMonkey<\/a> and <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=669730\">Firebug<\/a>.\u00a0 (They are the top four add-ons on <a href=\"http:\/\/addons.mozilla.org\/\">addons.mozilla.org<\/a>!)<\/li>\n<li>We have found leaks in simple, innocuous add-ons, such as <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=672619\">It&#8217;s All Text<\/a> and <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=691102\">Image Zoom.<\/a><\/li>\n<li>We have found leaks in add-ons written by large software companies, such as <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=727938\">McAfee SiteAdvisor<\/a> and <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=731801\">Yahoo! Toolbar<\/a>.<\/li>\n<\/ul>\n<p>Happily, most of the leaks in the mentioned add-ons have been fixed by their developers.\u00a0 But by now it should be clear that (a) memory leaks are common, and (b) no add-ons are immune, no matter how simple they are, or how good their developers are.<\/p>\n<p>The other thing to note is that these add-ons each exhibited a particular kind of leak that we call a <em>zombie compartment<\/em>.\u00a0 The <a href=\"https:\/\/developer.mozilla.org\/en\/Zombie_Compartments\">gory details of zombie compartments<\/a> are beyond the scope of this blog post;\u00a0 the important thing is that they are common, accounting for 90% or more of the add-on leaks we&#8217;ve seen.<\/p>\n<h4>Leaky Add-ons Make Firefox Slow and Crashy<\/h4>\n<p>So, leaky add-ons are common.\u00a0 What effect do they have?<\/p>\n<p>Not all leaks are created equal.\u00a0 Some add-ons leak a small amount of memory; some add-ons leak an unbounded amount of memory.\u00a0 Users with multiple add-ons are more likely to suffer problems. Certain browsing patterns may exacerbate leaks.\u00a0 Overall, it&#8217;s very hard to characterize.<\/p>\n<p>Nonetheless, affected users may experience high memory consumption, which has the following visible effects.<\/p>\n<ul>\n<li>Long freezes, caused by garbage collection and cycle collection.<\/li>\n<li>General sluggishness during basic operations, like scrolling and switching tabs.<\/li>\n<li>Occasional drastic slowdowns &#8212; particularly on machines with small amounts of RAM such as netbooks &#8212; due to the <a href=\"http:\/\/en.wikipedia.org\/wiki\/Paging\">paging<\/a> of data to and from the hard disk.<\/li>\n<li>Out-of-memory crashes or aborts, particularly on Windows and ARM machines.<\/li>\n<\/ul>\n<p>For example, one user who disabled a leaky version of 1Password experienced <a href=\"http:\/\/support.agilebits.com\/discussions\/browser-extensions\/1257-firefox-performance-degrades-with-extension\"><strong>a much leaner and faster Firefox<\/strong><\/a> as a result.<\/p>\n<blockquote><p>I only use a handful of extensions and honestly never suspected 1P, however after disabling it I noticed my FireFox performance increased very noticibly. I\u2019ve been running for 48 hours now without the 1P extension in Firefox and wow what a difference. <strong>Browsing is faster, switching is faster, memory usage is way down.<\/strong><\/p><\/blockquote>\n<p>(My emphasis.)<\/p>\n<p>We&#8217;ve heard this kind of story many times, and this one is typical because the user never suspected that an add-on was the cause of the problem.\u00a0 Leaks in add-ons are not Mozilla\u2019s fault, but they are Mozilla\u2019s problem &#8212; Firefox gets blamed for the sins of its add-ons.\u00a0 This isn&#8217;t surprising, because Firefox hasn&#8217;t provided tools that would let users understand when add-ons are causing problems.<\/p>\n<h4>What has Mozilla done about this before now?<\/h4>\n<p>For a long time, we lacked memory profiling tools that were (a) powerful enough to detect add-on leaks, and (b) simple enough to be used by users who weren&#8217;t also Firefox developers.\u00a0 This meant that we didn&#8217;t appreciate how many problems were caused by leaky add-ons.<\/p>\n<p>That changed last year.\u00a0 The infrastructure with which we detect zombie compartments was implemented in two parts:\u00a0 the segmentation of the JavaScript heap into <a href=\"http:\/\/andreasgal.com\/2010\/10\/13\/compartments\/\">compartments<\/a> (which happened in Firefox 4, released in March 2011), and the addition of <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=661474\">per-compartment memory reporting<\/a> in Firefox&#8217;s about:memory page (which was added in Firefox 7, released in September 2011).<\/p>\n<p>Once these tools were in place, the reports of leaky add-ons started rolling in, and it only took a few months to realize the magnitude of the problems they were causing.\u00a0 We started <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=700547\">tracking them<\/a>, we helped authors <a href=\"https:\/\/developer.mozilla.org\/en\/Extensions\/Common_causes_of_memory_leaks_in_extensions\">fix them<\/a>, and tightened the <a href=\"https:\/\/wiki.mozilla.org\/AMO:Editors\/EditorGuide\/AddonReviews\">AMO review guidelines<\/a> so that any add-on that was found to cause zombie compartments would have its status downgraded.<\/p>\n<p>As a result, dozens of leaky add-ons were fixed.\u00a0 But there are literally thousands of Firefox add-ons in existence, many of which aren&#8217;t hosted on AMO and so aren&#8217;t subject to the AMO review policies.\u00a0 And although many add-on authors were responsive, some weren&#8217;t.\u00a0 It was a tedious game of whack-a-mole, and something better was needed.<\/p>\n<h3>The Fix<\/h3>\n<p>Leaky add-ons usually cause zombie compartments by holding references to DOM structures within a web page even after the page has been closed or navigated away from.\u00a0 This prevents Firefox from garbage-collecting the page&#8217;s compartment.<\/p>\n<p>Mozilla engineer Kyle Huey had the radical idea of simply <a href=\"http:\/\/blog.kylehuey.com\/post\/21892343371\/fixing-the-memory-leak\">detecting these kinds of references and cutting them<\/a>.\u00a0 This would eradicate this entire class of memory leaks for good, without requiring any changes to individual add-ons.<\/p>\n<p>However, it wasn&#8217;t at all obvious that this would work.\u00a0 In particular, it was possible that code within Firefox and\/or add-ons might be broken by this change.\u00a0 But it turned out, once some <a href=\"http:\/\/blog.mozilla.org\/nnethercote\/2012\/05\/15\/additional-update-on-leaky-add-ons\/\">bumps<\/a> were cleared, that almost no code was broken and the leaks were prevented exactly as predicted.\u00a0 This fix is present in Firefox 15.<\/p>\n<h4>Confirmation via in-house testing<\/h4>\n<p>I did some early testing on a number of add-ons that were known to leak:\u00a0 <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=669730\">Firebug 1.9.1<\/a>,<a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=727938\"> McAfee SiteAdvisor 3.4.1<\/a>, <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=735401\">PicPasso 1.0.1<\/a>, <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=740085\">LoL 1.5<\/a>, <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=742232\">YouTube Ratings Preview 1.03<\/a>, <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=745149\">Enter Selects 7<\/a>, <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=747167\">HttpFox 0.8.10<\/a>, and <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=728589\">Link Resolve 0.7<\/a>.\u00a0 In every case, Firefox 15 prevents the leaks.<\/p>\n<p>We&#8217;ve subsequently confirmed that a number of other leaking add-ons have been fixed by this change.\u00a0 Although add-ons can still cause zombie compartments in obscure cases, we are yet to encounter such a case in practice.<\/p>\n<h4>Confirmation from real-world users<\/h4>\n<p>Firefox users can enable <a href=\"http:\/\/blog.mozilla.org\/tglek\/2011\/09\/20\/firefox-7-telemetry-faster-startup\/\"><em>telemetry<\/em><\/a>, which is a feature that periodically sends anonymous performance-related data to Mozilla.\u00a0 This allows us to analyze how people use Firefox in the real world.<\/p>\n<p>We don&#8217;t track zombie compartments via telemetry, but we do track something called <em>ghost windows<\/em>.\u00a0 The <a href=\"http:\/\/jlebar.com\/2012\/5\/30\/A_ghost_story.html\">gory details of ghost windows<\/a> are also beyond the scope of this blog post;\u00a0 the important thing to know is that they correlate strongly with zombie compartments &#8212; a high ghost window count very likely indicates the presence of zombie compartment, and thus memory leaks.<\/p>\n<p>The following graph shows the ghost windows telemetry measurements for users of Firefox Nightly builds on Windows during April and May 2012.\u00a0 The line marked with a <strong>1<\/strong> shows when Kyle&#8217;s patch landed, and the lines marked with <strong>2<\/strong> and <strong>3<\/strong> show when some related follow-up patches landed.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone\" src=\"http:\/\/jlebar.com\/2012\/ghost-windows-plot.png\" alt=\"Ghost Windows telemetry results\" width=\"524\" height=\"377\" \/><\/p>\n<p>Telemetry data tends to be very noisy, and this graph is no exception, but the effect is clear.\u00a0 The mean number of ghost windows (shown by the blue dots) dropped from typically between three and four to typically less than one.\u00a0 (As for why it&#8217;s not zero, one\u00a0reason is that ghost windows can be false positives &#8212; i.e. they don&#8217;t always correspond to actual leaks.)\u00a0 Furthermore, the variance has dropped greatly.\u00a0 Previously, the hardest-hit 5% of measurements would typically have at least 15 or 20 ghost windows, and the next hardest-hit 20% would typically have at least two.\u00a0 After the fix, the hardest-hit 5% of measurements typically have only two or more ghost windows, and the remaining 95% typically have zero.\u00a0 This graph is a few weeks old now, but the measurements for Firefox 15 since then have remained steady.<\/p>\n<p>In other words,\u00a0ghost windows have changed from something that many users experience to something that few users experience.\u00a0 Furthermore, those users who are still affected experience many fewer than before.\u00a0 This graph also provides further evidence that add-ons are responsible for a large fraction of the memory leaks encountered by Firefox users.<\/p>\n<h3>The Benefits of the Fix<\/h3>\n<h4>Our testing<\/h4>\n<p>I compared pre-release versions of Firefox 14 and Firefox 15 with McAfee SiteAdvisor 3.4.1 installed.\u00a0 SiteAdvisor is a very popular add-on that rates the safety of web sites, and version 3.4.1 had an unbounded memory leak that leaked significant amounts of memory on <em>every page that Firefox visited<\/em>.\u00a0 (That leak has since been fixed and version 3.4.1 is no longer available from McAfee.)<\/p>\n<p>I did the testing on a fast desktop machine with Windows 7 installed.\u00a0 I ran <a href=\"http:\/\/gregor-wagner.com\/tmp\/mem\">MemBench<\/a>, which is a memory test benchmark that opens 150 popular websites, one per tab, and then closes them all.\u00a0 This test is a reasonable facsimile of several hours of web browsing.<\/p>\n<p>I used about:memory to measure memory consumption (a) with the 150 tabs open (plus one more tab for about:memory itself), and then (b) after closing those 150 tabs.\u00a0 The metric I measured was <a href=\"http:\/\/en.wikipedia.org\/wiki\/Resident_set_size\"><em>resident<\/em><strong><\/strong><\/a> memory consumption, which is the amount of physical memory used by Firefox.\u00a0 It is measured by the operating system.\u00a0 Firefox 14&#8217;s two measurements are on the left, and Firefox 15&#8217;s two measurements are on the right.<\/p>\n<p><a href=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2012\/07\/resident2.png\"><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-large wp-image-2111\" title=\"resident\" src=\"http:\/\/blog.mozilla.org\/nnethercote\/files\/2012\/07\/resident2-1024x856.png\" alt=\"Resident Memory Consumption (MB) on MemBench of Firefox 14 &amp; 15 with SiteAdvisor 3.4.1 add-on\" width=\"500\" height=\"417\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2012\/07\/resident2-1024x856.png 1024w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2012\/07\/resident2-300x250.png 300w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2012\/07\/resident2.png 1106w\" sizes=\"(max-width: 500px) 100vw, 500px\" \/><\/a><\/p>\n<p>The graph tells a clear story.\u00a0 The leak in SiteAdvisor 3.4.1 prevents Firefox 14 from releasing most of the memory used by the 150 MemBench tabs, even after those tabs are closed.\u00a0 In contrast, Firefox 15 prevents the leak.\u00a0 As a result,<strong> after closing 150 tabs, Firefox 15&#8217;s resident memory consumption is 4.8x smaller than Firefox 14&#8217;s<\/strong> (compare the red bars) <strong>without any change to the add-on<\/strong>.<\/p>\n<h4>Feedback from users<\/h4>\n<p>The feedback from users of the pre-release <a href=\"http:\/\/nightly.mozilla.org\/\">Nightly<\/a> and <a href=\"http:\/\/nightly.mozilla.org\/\">Aurora<\/a> versions of Firefox has been overwhelmingly positive.\u00a0 Here are some notable examples.<\/p>\n<p>One user experienced<strong> <a href=\"http:\/\/blog.kylehuey.com\/post\/21892343371\/fixing-the-memory-leak#comment-519424145\">a 4x reduction in memory consumption at the end of the day<\/a><\/strong>.<\/p>\n<div id=\"dsq-comment-message-519424145\">\n<div id=\"dsq-comment-text-519424145\">\n<blockquote><p>I seem to be seeing really significant improvements from this. Normally Fx would start at 170MB and by the end of the day that would often be creeping up to 800 or 900 MB. Today using latest nightly I again started at 170, but now at the end of the day I&#8217;m only at 230MB!<\/p>\n<p>I knew that my addons were the culprit, yet the list I have at the moment is a tenured list of addons I wish to keep. This fix allows me to continue being an addon junkie without the pernicious side effects. Thank you so much!<\/p><\/blockquote>\n<\/div>\n<\/div>\n<p>Another user reported via email that\u00a0<strong>a 1.5GB per hour leak &#8212; which caused 10+ second pauses and necessitated frequent restarts &#8212; disappeared<\/strong>.<\/p>\n<blockquote><p>Firefox was leaking about 1.5GB per hour for me. It started with Firefox 3. I tracked it down to Ghostery and NoScript, but even without those addons it leaked about 500MB per hour of browsing.<\/p>\n<p>GC and CC times got up into the 10 second range. Ugly. Really really ugly! And this is on top of the line massively overclocked hardware, too. I had to install a new addon to add a restart button to Firefox, because Firefox froze solid after hitting 2GB of memory usage&#8230;<\/p>\n<p>Then your patch comes along and solves it all\u2026 totally awesome!<\/p><\/blockquote>\n<p>Another user experienced <strong><a href=\"http:\/\/blog.kylehuey.com\/post\/21892343371\/fixing-the-memory-leak#comment-524693082\">faster painting, scrolling, and input operations<\/a><\/strong>.<\/p>\n<blockquote><p>Certainly, before this fix I would find that Firefox often became sluggish (input lag, slow paint operations, less than silky smooth scroll animations) as the memory usage built up&#8230; I haven\u2019t noticed the same symptoms since&#8230; the signs certainly seem to be good.<\/p><\/blockquote>\n<p>These quotes nicely highlight the potential benefits:\u00a0 greatly reduced memory consumption, which leads to fewer pauses, faster overall performance, and no need to restart Firefox to restore optimum performance.<\/p>\n<h4>YMMV<\/h4>\n<p>Based on the evidence above, we are confident that Firefox 15 fixes the vast majority of add-on memory leaks, and that as a result, many users will see drastic improvements in Firefox&#8217;s performance and stability.\u00a0 However, predicting the exact effects of this fix on the hundreds of millions of Firefox users is essentially impossible, due to the huge number of different add-ons that are available, and the great variety of add-on and browsing usage patterns.<\/p>\n<p>Nonetheless, if you are a heavy add-on user and you have noticed Firefox&#8217;s performance degrade during long browsing sessions, there is a good chance you will see large improvements.\u00a0 Firebug users may notice improvements as well.\u00a0 But, as they say:\u00a0 your mileage may vary.<\/p>\n<h3>CONCLUSION<\/h3>\n<p>Over the past year, Mozilla has made great progress in reducing Firefox&#8217;s memory consumption.\u00a0 However, the excessive memory consumption caused by add-ons with memory leaks has remained an ongoing problem.<\/p>\n<p>Firefox 15 fixes that problem.\u00a0 We have confirmed, via in-house testing and from real-world telemetry data, that it prevents the vast majority of leaks that occur in existing add-ons.<\/p>\n<p>Users who upgrade to Firefox 15 won&#8217;t have to upgrade their add-ons to see the benefits.\u00a0 While it is hard to predict the effect of this improvement on any individual user, many users should experience greatly reduced memory consumption, particularly on long browsing sessions.\u00a0 This should result in better performance, fewer pauses, and fewer crashes.<\/p>\n<p>Mozilla&#8217;s MemShrink efforts are ongoing.\u00a0 We have various projects in the pipeline that aim to further reduce Firefox&#8217;s memory consumption, and help users understand better how Firefox is using memory.\u00a0 Hopefully these changes will make it into Firefox releases not far from now!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>TL;DR Firefox 15 prevents most memory leaks caused by add-ons, including Firebug.\u00a0 For many users with add-ons installed this will significantly reduce Firefox&#8217;s memory consumption, without requiring upgrades to those add-ons. For those users, Firefox 15 is likely to be faster (sometimes drastically so) and less likely to crash, especially if they have multiple add-ons [&hellip;]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[119,30,4544,4546],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/2020"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=2020"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/2020\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=2020"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=2020"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=2020"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}