I changed things so that JavaScript memory consumption for web content is reported on a per-tab basis<\/a>, as the following example shows.<\/p>\n All the major components of each tab’s memory consumption — JS, layout, style sheets, and DOM — are now reported on a per-tab basis.\u00a0 This is great, because it’s something that people have been requesting for\u00a0 years.\u00a0 However, the presentation is still intimidating.\u00a0 Consider the following excerpt from about:memory.<\/p>\n Among those From a user’s point of view, the visible things they have control over (i.e. windows and tabs) should be better distinguished from everything else, and be shown with nicer names than “browser.xul”.\u00a0 But the architecture of Firefox’s internals doesn’t make this easy.\u00a0 I’ve made one not-very-successful attempt<\/a> to improve this, and I’d love to hear suggestions from other Firefox devs on how to present this information better.<\/p>\n (The step after that one is to present some kind of “tab manager” — a massively stripped-down variant of about:memory that ordinary users can understand.\u00a0 This would also require localization for all the languages that Firefox supports.\u00a0 One step at a time!)<\/p>\n Finally, another memory reporting improvement happened:\u00a0 Andrew McCreight and I added measurement of orphan DOM nodes<\/a>.\u00a0 These are DOM nodes that have been discarded by a page but are still accessible from JavaScript objects.\u00a0\u00a0 They were the biggest single remaining contributor to “heap-unclassified”.\u00a0 For example, when I hit the “update” button on about:memory I get 5MB of them (don’t worry, they go away when the garbage collector runs).\u00a0 Also, badly coded sites can create large numbers of them.\u00a0 So it’s a good thing to be able to measure, for both users and web developers.<\/p>\n Mihai Sucan fixed a bad leak involving pages that use Wladimir Palant fixed an issue in AdBlock Plus that causes its memory consumption to slowly grow when viewing sites that repeatedly reset the src property of images, such as Mibbit<\/a>.\u00a0 (The fine-grained memory reporting enabled by compartment-per-global also made this diagnosis much easier.)\u00a0 This fix is in Adblock Plus 2.1.<\/p>\n Alexandre Poirot fixed a minor leak in the Add-on SDK<\/a>.<\/p>\n I wrote about the difficulties of cross-browser memory comparisons<\/a>.<\/p>\n Here are the current bug counts.<\/p>\n A net reduction of seven bugs!<\/p>\n In January, I described six “big ticket items<\/a>” that needed tackling to improve Firefox’s memory consumption.\u00a0 Let’s look at how they’ve progressed since then.<\/p>\n This had two parts.\u00a0 The first part was to only generate bytecode for a function once it is run<\/a>.\u00a0 I’ve made a good chunk of progress towards and have it working on very simple examples.\u00a0 This has required a great deal of refactoring of the SpiderMonkey front-end<\/a>.\u00a0 (The front-end is a hairy piece of code, so this is virtuous in and of itself.)\u00a0 There are still a couple of other<\/a> bugs<\/a> blocking it from further progress.\u00a0 I intend to return to continue on this once they are done.<\/p>\n The second part was to share immutable parts of scripts between web pages<\/a>.\u00a0 No progress has been made on this.<\/p>\n So, overall, this item is about 25% done, and progress is still ticking along slowly.<\/p>\n This one also had two parts.\u00a0 The first part was to reduce about:memory’s \u201cheap-unclassified\u201d number (a.k.a \u201cdark matter\u201d) to typically 10%<\/a>.\u00a0 People such as Nathan Froyd and I have made good progress on this.\u00a0 It’s now occasionally below 10% (my current session shows 7.5%) but often is around 15%.\u00a0 That’s much better than the 20–25% we typically were getting at the start of the year, and it’s low enough that people rarely complain about it now.\u00a0 And we’re very much into the long tail now, so it’ll be hard to improve things much further.\u00a0 For the most part I don’t think we need to.<\/p>\n The second part was to report memory in a per-tab fashion.\u00a0 As mentioned above, I just completed this, although the presentation is not as user-friendly as it could be.<\/p>\n The per-tab reporting was enabled by compartment-per-global.\u00a0 Compartment-per-global also gives us much more insight into the memory consumption of Firefox’s own JavaScript code, and even some idea about add-on memory consumption, which was (to me) an unexpected bonus.\u00a0 Indeed, its usefulness was demonstrated by the two leak fixes mentioned above.<\/p>\n So, overall, this item is about 80% done.\u00a0 Although there is still some work to be done, it has progressed enough that I will remove it from the “big ticket items” list.<\/p>\n We can cross this item off the list, thanks to areweslimyet.com<\/a>.\u00a0 In January I also mentioned the idea of using telemetry for this, but we’ve found that telemetry results are so noisy that little (with rare exceptions<\/a>) can be gleaned from it, so there’s not really anything to be done on that front.<\/p>\n This is one of the two key SpiderMonkey performance features under development (the other being IonMonkey<\/a>), and so people such as Terrence Cole, Steve Fink, Brian Hackett, and Bill McCloskey and actively working on it.\u00a0 Unfortunately, it’s a huge undertaking that requires rewriting many of SpiderMonkey’s internal and external APIs.\u00a0 If I had to guess I’d say it’s 25% complete, but that really is just a wild guess.\u00a0 I’ll be pleased if it is finished before the end of the year.<\/p>\n This is the big ticket item for which the least progress has been made. There’s a single bug<\/a> that is blocking all the follow-on work.\u00a0 Both Timothy Nikkel and Jet Villegas have spent time working on it, but it has been a difficult nut to crack.\u00a0 Justin Lebar had a recent suggestion<\/a> for narrowing its scope slightly.\u00a0 I hope it’s a good suggestion, because I’m out of ideas on this one.<\/p>\n This is the most satisfying item of the lot.\u00a0 I expected to make only grudging, piecemeal progress, but Kyle Huey’s fix to prevent zombie compartments<\/a> prevents the vast majority of add-on leaks, enough to declare victory and cross this item off the list.\u00a0 I’ll be writing more about this next week.<\/p>\n Three items (#1, #4, #5) have progressed enough that they can be removed from the list.\u00a0 Only one of these directly improved Firefox’s memory consumption;\u00a0 the other two were about better measuring and monitoring of memory consumption.<\/p>\n The remaining items are about reducing memory consumption.\u00a0 Two of them (#3, #6) have a long way to go, but progress has been made and is ongoing.\u00a0 One item (#2) has made little progress and has stalled.<\/p>\n Here’s my updated list of the most important things.<\/p>\n As mentioned above, this has two parts: lazy bytecode generation<\/a>, and the\u00a0 sharing of immutable parts of scripts<\/a>.\u00a0 I intend to continue working on the first part.\u00a0 Once that’s done, it may turn out that the second part isn’t necessary;\u00a0 we’ll see.<\/p>\n The measurements on areweslimyet.com have been creeping up for the past two months.\u00a0 The major cause is compartment-per-global, which has lots of benefits (as mentioned above) but also increases memory consumption by a small-to-moderate amount across the board.\u00a0 This is because there are many more compartments than there used to be, and there’s a certain amount of overhead and waste in each compartment.<\/p>\n Generational GC will help, but there are some other things that can be done more quickly that will help.\u00a0 See the tracking bug<\/a> for one example.<\/p>\n Boot2Gecko (now officially known as “Firefox OS”) is going to be running on low-end phones without much physical memory, so that’ll require care.\u00a0 The first step is to get about:memory working on B2G<\/a>.\u00a0 The next step is to find a way to copy the data from about:memory somewhere else<\/a>, because B2G doesn’t have cut and paste!\u00a0 Beyond that, it’s unclear;\u00a0 but once about:memory is working it’s very likely it will identify some sub-optimal B2G-specific behaviour.<\/p>\n Let me quote myself<\/a> to explain the memory consumption benefits of compacting, generational GC.<\/p>\n Virtual memory consumption drops in two ways.\u00a0 First, the compaction minimizes waste due to fragmentation.\u00a0 Second, the heap grows more slowly.\u00a0 Physical memory consumption drops for the same two reasons.<\/p><\/blockquote>\n The performance benefits are also significant.<\/p>\n Performance improves for three reasons.\u00a0 First, paging is reduced because of the generational behaviour:\u00a0 much of the JS engine activity occurs in the nursery, which is small;\u00a0 in other words, the memory activity is concentrated within a smaller part of the working set.\u00a0 Second, paging is further reduced because of the compaction: this reduces fragmentation within pages in the tenured heap, reducing the total working set size.\u00a0 Third, the tenured heap grows more slowly because of the generational behaviour… which means that structure traversals done by the garbage collector (during full-heap collections) and cycle collector are faster.<\/p><\/blockquote>\n Important stuff.<\/p>\n On image-heavy pages, Firefox uses much more memory than other browsers.\u00a0 This includes things like Facebook image slideshows.\u00a0 One commenter said<\/a> the following.<\/p>\n On behalf of Facebook Inc, let me just say that we are prepared to give over one thousand free “pokes” to anyone who fixes this bug or alternatively https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=683290<\/a>. I will also gladly show you Mark Zuckerberg’s desk and give you free dinner and cookies as part of a complementary tour of Facebook HQ in Menlo Park.<\/p>\n<\/blockquote>\n I have no idea if this person is really from Facebook, but it’s certainly suggestive.\u00a0 Also, I’ve heard from B2G people that they are having to work around this problem in their Gallery app.<\/p>\n To repeat myself from week 32:\u00a0 there are three MemShrink:P1 bugs relating to this:\u00a0 one about not decoding all images immediately<\/a>, one about discarding non-visible decoded images after some time<\/a>, and one about some infrastructure work<\/a> that is required for the first two.\u00a0 (See this discussion<\/a> on the dev-platform mailing list for more details about this topic.)<\/p>\n There are three repeat items, and two new items, but there are fewer items than last time;\u00a0 this reflects the fact that we are in a better position than we were in January.\u00a0 If you think I’ve overestimated or underestimated the importance of any of these, or omitted anything, I’d be interested to hear.<\/p>\n One final thing:\u00a0 last time, several people suggested “better reporting of add-on memory consumption”.\u00a0 If I knew of any ideas on how to do this better than about:memory currently does, I’d gladly put it on the list.\u00a0 But I don’t, and there doesn’t seem to be much point to put an item on the list that we don’t know how to implement.<\/p>\n I’m taking the next two weeks as vacation, so there won’t be a MemShrink report on July 24.\u00a0 I’ll be back with the next MemShrink round-up on August 8th.\u00a0 See you then.<\/p>\n","protected":false},"excerpt":{"rendered":" Memory Reporting I changed things so that JavaScript memory consumption for web content is reported on a per-tab basis, as the following example shows. \u2502\u00a0 \u251c\u2500\u2500\u2500\u25004,472,568 B (00.50%) — top(http:\/\/www.mozilla.org\/en-US\/firefox\/fx\/, id=236)\/active \u2502\u00a0 \u2502\u00a0\u00a0\u00a0 \u251c\u2500\u25004,192,640 B (00.47%) — window(http:\/\/www.mozilla.org\/en-US\/firefox\/fx\/) \u2502\u00a0 \u2502\u00a0\u00a0\u00a0 \u2502\u00a0 \u251c\u2500\u25001,979,112 B (00.22%) ++ js\/compartment(http:\/\/www.mozilla.org\/en-US\/firefox\/fx\/) \u2502\u00a0 \u2502\u00a0\u00a0\u00a0 \u2502\u00a0 \u251c\u2500\u25001,607,216 B (00.18%) ++ layout \u2502\u00a0 […]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4550,119,30,4555,4544,4546],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/2140"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=2140"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/2140\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=2140"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=2140"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=2140"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\u2502\u00a0 \u251c\u2500\u2500\u2500\u25004,472,568 B (00.50%) -- top(http:\/\/www.mozilla.org\/en-US\/firefox\/fx\/, id=236)\/active\r\n\u2502\u00a0 \u2502\u00a0\u00a0\u00a0 \u251c\u2500\u25004,192,640 B (00.47%) -- window(http:\/\/www.mozilla.org\/en-US\/firefox\/fx\/)\r\n\u2502\u00a0 \u2502\u00a0\u00a0\u00a0 \u2502\u00a0 \u251c\u2500\u25001,979,112 B (00.22%) ++ js\/compartment(http:\/\/www.mozilla.org\/en-US\/firefox\/fx\/)\r\n\u2502\u00a0 \u2502\u00a0\u00a0\u00a0 \u2502\u00a0 \u251c\u2500\u25001,607,216 B (00.18%) ++ layout\r\n\u2502\u00a0 \u2502\u00a0\u00a0\u00a0 \u2502\u00a0 \u251c\u2500\u2500\u2500\u2500352,520 B (00.04%) \u2500\u2500 style-sheets\r\n\u2502\u00a0 \u2502\u00a0\u00a0\u00a0 \u2502\u00a0 \u251c\u2500\u2500\u2500\u2500253,312 B (00.03%) ++ dom\r\n\u2502\u00a0 \u2502\u00a0\u00a0\u00a0 \u2502\u00a0 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500480 B (00.00%) \u2500\u2500 property-tables<\/pre>\n
\u251c\u2500\u2500\u250018,288,696 B (16.43%) -- window-objects\r\n\u2502\u00a0\u00a0 \u251c\u2500\u2500\u25006,263,208 B (05.62%) ++ top(about:memory?verbose, id=6)\/active\r\n\u2502\u00a0\u00a0 \u251c\u2500\u2500\u25005,090,352 B (04.57%) ++ top(chrome:\/\/browser\/content\/browser.xul, id=1)\/active\r\n\u2502\u00a0\u00a0 \u251c\u2500\u2500\u25003,422,280 B (03.07%) ++ top(http:\/\/www.mozilla.org\/en-US\/firefox\/fx\/, id=17)\/active\/window(http:\/\/www.mozilla.org\/en-US\/firefox\/fx\/)\r\n\u2502\u00a0\u00a0 \u251c\u2500\u2500\u25003,178,432 B (02.85%) ++ top(chrome:\/\/global\/content\/console.xul, id=21)\/active\r\n\u2502\u00a0\u00a0 \u2514\u2500\u2500\u2500\u2500\u2500334,424 B (00.30%) ++ top(resource:\/\/gre-resources\/hiddenWindow.html, id=3)\/active\/window(resource:\/\/gre-resources\/hiddenWindow.html)<\/pre>\n
top(...)<\/code> entries are two browser tabs (one for www.mozilla.org, one for about:memory), two browser chrome windows (browser.xul is the main browser window, and console.xul is the error console window), and the hidden window which is always present.\u00a0 Firefox developers can easily understand this, and technically-inclined users may be able to guess what’s going on, but the average user will struggle.<\/p>\nMiscellaneous<\/h3>\n
window.console<\/code><\/a>.\u00a0 Multiple people reported that this caused Firefox’s memory consumption to balloon to multiple GB.\u00a0 This leak was introduced during the Firefox 15 development cycle, and the fix has been backported to the Aurora channel, which is where Firefox 15 currently resides.\u00a0 One notable thing about this fix is that the problem was quite easy to diagnose because about:memory clearly indicated that the ConsoleAPI.js compartment was responsible for most of the memory consumption.\u00a0 Without the fine-grained memory reporting that was enabled by compartment-per-global<\/a>, that memory would have been reported in a generic “System Principal” compartment which would have made diagnosis much<\/em> harder.<\/p>\nBug Counts<\/h3>\n
\n
Update on The big ticket Items<\/h3>\n
#6: Better Script Handling<\/h4>\n
#5: Better Memory Reporting<\/h4>\n
#4: Better Memory Consumption Tracking<\/h4>\n
#3: Compacting Generational GC<\/h4>\n
#2: Better Foreground Tab Image Handling<\/h4>\n
#1: Better Detection and Notification of Leaky Add-ons<\/h4>\n
Summary<\/h4>\n
The New big ticket Items<\/h3>\n
#5: Better Script Handling<\/h4>\n
#4: Regain compartment-per-global losses<\/h4>\n
#3: Boot2Gecko<\/h4>\n
#2: Compacting Generational GC<\/h4>\n
#1: Better Foreground Tab Image Handling<\/h4>\n
\n
Summary<\/h4>\n
Vacation<\/h3>\n