{"id":822,"date":"2011-05-25T13:05:44","date_gmt":"2011-05-25T02:05:44","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=822"},"modified":"2011-05-25T13:05:44","modified_gmt":"2011-05-25T02:05:44","slug":"a-new-way-of-tracking-leak-reports","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2011\/05\/25\/a-new-way-of-tracking-leak-reports\/","title":{"rendered":"A new way of tracking leak reports"},"content":{"rendered":"<p>We get lots of leak reports from users.\u00a0 There is a spectrum of quality.<\/p>\n<ul>\n<li>Some are hopelessly vague and will never lead to anything useful. (&#8220;After browsing for several hours, Firefox is using 100s of MBs of memory.\u00a0 This is unacceptable;\u00a0 please fix.&#8221;)\u00a0 <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=643177\">Bug 643177<\/a> is an example.<\/li>\n<li>Some are very precise.\u00a0 This makes them easy to reproduce, likely to be fixed quickly, and easy to re-confirm if other leaks are fixed in the interim.\u00a0 (&#8220;I managed to reduce the problem down to the attached 10 line HTML file, it causes my machine to run out of memory within 10 seconds of loading.&#8221;) <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=654106\">Bug 654106<\/a> is a good example.<\/li>\n<li>Most are somewhere between these two extremes.<\/li>\n<\/ul>\n<p>Because many of the reports aren&#8217;t great, it can be hard to tell if the problem is still present some time later. A single leak may be reported N times, then fixed, and N-1 reports stay open.\u00a0 In short, leak reports get stale.\u00a0 (This is true of many bug reports, but I think leak reports are more prone to staleness than most.)<\/p>\n<h3>How bugs are currently tracked<\/h3>\n<p>There is a keyword, &#8216;mlk&#8217;, which is added to almost all leak reports.\u00a0 There are over 600 open bugs with that keyword, going back over 10 years.\u00a0 So it&#8217;s not much use.<\/p>\n<p>In the lead-up to Firefox 4, I used <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=632234\">bug 632234<\/a> (which I&#8217;ll henceforth call &#8220;mlk-fx4-old&#8221;) to track potentially blocking leaks.\u00a0 It worked well.<\/p>\n<p>After that, I created <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=640452\">bug 640452<\/a> (which I&#8217;ll henceforth call &#8220;mlk-fx5+&#8221;), with which I&#8217;ve been tracking leaks in the lead-up to Firefox 5 and later versions.\u00a0 I carried over unresolved bugs from mlk-fx4-old.\u00a0 mlk-fx5+ is starting to fill up feel stale.\u00a0 Basically, I can see it suffering the same problems as the &#8216;mlk&#8217; keyword before too long.<\/p>\n<p>So I&#8217;m thinking about changing how these are tracked.\u00a0 The basic idea is to use keep using the &#8216;mlk&#8217; keyword for all leak reports, and then have one leak-tracking bug for each version of Firefox, so it&#8217;s clear which version each report applies to.<\/p>\n<h3>Steps needed to start this<\/h3>\n<p>Add the &#8216;mlk&#8217; keyword to all mlk-fx4-old and mlk-fx5+ bugs that lack it.<\/p>\n<p>Open new tracking bugs: mlk-fx4-beta, mlk-fx4, mlk-fx5, mlk-fx6.\u00a0 (The Firefox 4 beta period was long enough, and there were enough leak reports filed against beta versions, that separating mlk-fx4-beta and mlk-fx4 seems worthwhile.)\u00a0 Make each mlk-fxN depend on mlk-fx(N-1).<\/p>\n<p>For all the existing bugs tracked by mlk-fx4-old and mlk-fx5+, add them to the appropriate new tracking bug.\u00a0 With one exception: for hopelessly vague ones, just mark them as duplicates of mlk-fxN, with an explanatory message (&#8220;we&#8217;re not ignoring leaks, look at all these ones we&#8217;re tracking!\u00a0 but your report doesn&#8217;t tell us anything we don&#8217;t already know, sorry&#8221;).<\/p>\n<p>Close mlk-fx5+.<\/p>\n<h3>Steps needed to maintain this in the future<\/h3>\n<p>When Firefox version N&#8217;s cycle starts, open mlk-fxN, and mark it as depending on mlk-fx(N-1).<\/p>\n<p>For all new leak reports, mark it as blocking mlk-fxN, for appropriate N.\u00a0 Also add the &#8216;mlk&#8217; keyword.<\/p>\n<p>If someone confirms in a comment that a problem reported in version N is still present in version N+1, mark that bug as also blocking mlk-fx(N+1).<\/p>\n<h3>Properties of this system<\/h3>\n<p>You can still search for all leak reports, based on the &#8216;mlk&#8217; keyword.<\/p>\n<p>You can immediately tell roughly how stale a report is likely to be, based on which mlk-fxN tracking bug it blocks.\u00a0 This is more reliable than the bug number or file date;\u00a0 for example, we are still getting reports against Firefox 4 even though Firefox 5 (which has fixed a number of leaks) is in beta and Firefox 6 just went to Aurora.\u00a0 This immediately gives a starting priority for all leak reports:\u00a0 more recent ones have higher priority because they&#8217;re more likely to still be unfixed.<\/p>\n<p>Hopelessly vague reports are resolved immediately by duplicating, so they don&#8217;t clog things up.<\/p>\n<p>Tracking bugs shouldn&#8217;t get too big and unwieldy, because each Firefox version has a limited lifespan.<\/p>\n<p>Reports against version N still block mlk-fx(N+1), but via one level of indirection.\u00a0 Reports against version N+2 still block mlk-fx(N+2), but via two levels of indirection, etc.\u00a0 So the full chain of dependencies is maintained.<\/p>\n<p>We could periodically go through older bugs (eg. 3 releases ago) and ask people to re-confirm, and close out ones that get no response.\u00a0 But we wouldn&#8217;t have to do that.<\/p>\n<h3>Am I crazy?<\/h3>\n<p>Is this bureaucratic overkill?\u00a0 I don&#8217;t think so.\u00a0 It&#8217;ll take some work, but I&#8217;m happy to do that.\u00a0 It&#8217;ll only take an hour or two to set up, and then it won&#8217;t be much harder to maintain than what I&#8217;m currently doing with the mlk-fx5+ bug.\u00a0 (I also have plans for writing instructions to help users file better leak reports.)\u00a0 And it&#8217;ll allow us to proceed much more usefully with the lists of leak reports that we have.<\/p>\n<p>But I&#8217;m interested to hear if you disagree, or have any ideas for improving it.\u00a0 Thanks!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We get lots of leak reports from users.\u00a0 There is a spectrum of quality. Some are hopelessly vague and will never lead to anything useful. (&#8220;After browsing for several hours, Firefox is using 100s of MBs of memory.\u00a0 This is unacceptable;\u00a0 please fix.&#8221;)\u00a0 Bug 643177 is an example. Some are very precise.\u00a0 This makes them [&hellip;]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[142,30,4544,4546],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/822"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=822"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/822\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=822"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=822"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=822"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}