MemShrink progress, week 67–68

It’s been a busy couple of weeks for MemShrink.

B2G Memory Reporting

about:memory is our main tool for understanding memory consumption. On desktop Firefox it works very well.  On Firefox mobile it works less well, partly because it’s hard to read all that text on a small screen, and partly because you can’t cut and paste, which makes bug reporting difficult.  And on B2G (a.k.a. Firefox OS) it currently doesn’t work at all.

To remedy this, I implemented the ability to import and export memory reporter data in JSON format, and Justin Lebar sped up the exporting of this data and implemented a signal-based mechanism for triggering it.  The idea is that developers can trigger an export of the data from their device and then view it in about:memory in desktop Firefox.

This will allow detailed memory profiling for B2G, which I have listed as #3 on the current MemShrink “big ticket items” list.  The timing is good, because B2G has reached feature-freeze and so focus will now move to its performance and memory consumption as part of “Operation Slim Fast“.

(On a related note, Kartikaya Gupta and other engineers have started looking closely at the memory consumption of Firefox mobile on low-end devices.)

Generic Memory Reporting

I added memory reporting to IonMonkey.  Happily, it appears to use very little memory, especially compared with JaegerMonkey.  This is partly because it generates smaller code than JaegerMonkey, but mostly it’s because IonMonkey is only used to compile the small fraction of code that has been executed many 1000s of times.  In comparison, JaegerMonkey is used to compile the much larger fraction of code that has been executed more than a few times.

Kyle Huey added memory reporters for attribute nodes and attribute maps, which improves the coverage of the memory reporting for the DOM.

Gecko Slimmings

Kyle Huey optimized the representation of repeated inline style rules.  This means that for pages with many elements of the form <foo style="blah">, the repeated style="blah" part is now shared between the elements.  This is a huge win on certain large, auto-generated pages.  For example, on one page that holds a gigantic Mercurial diff, this change reduced 64-bit Firefox’s memory consumption from 1,759 MiB to 1,306 MiB.

Kyle Huey and Olli Pettay fixed a leak relating to web workers.

Patrick McManus fixed a leak in network code that was causing intermittent oranges for Mochitests.

Seth Fowler made nsConsoleService only post a LogMessageRunnable to the event queue if there are any observers for it.  This can save a lot of memory for pages that contains many errors.

Bad Graphics Drivers

An ongoing problem we have is that badly-written graphics drivers can cause outlandish (e.g. multi-GB) memory consumption.  Here’s one example that was fixed by blocklisting one particular AMD driver.  (That driver was also causing lots of crashes.)

It’s got to the point that if someone is seeing excessive memory consumption with no apparent explanation — e.g. immediately after start-up, with few tabs open — I’ll suggest they set layers.acceleration.disabled to true in about:config and there’s a good chance it’ll fix the problem.  (Restarting in safe mode also disables hardware acceleration.)  I don’t have any good ideas on how to improve this situation.

Bug Counts

Here are the current bug counts.

  • P1: 16 (-8/+3)
  • P2: 97 (-2/+10)
  • P3: 97 (-6/+3)
  • Unprioritized: 3 (-3/+3)

In today’s MemShrink meeting we closed and downgraded a number of P1 bugs that (a) have been partially addressed, or (b) are less important than they first seemed, or (c) are duplicates.

17 Responses to MemShrink progress, week 67–68

  1. Maybe you can speak with somebody from Valve, now that the are working on a linux client the have some good info about the regular issues with video drivers on all platforms.

    In any case also maybe you can use some tools they use, thanks to phoronix http://www.phoronix.com/scan.php?page=news_item&px=MTE0NDE we know that they use http://sysprof.com/ and https://github.com/apitrace/apitrace at least for linux.

  2. Re: bad gfx drivers, we could keep track of roughly how much video memory we think we’re using, query video memory usage (hoping for the driver to return an accurate value) and if the ratio between the 2 values is higher than, say, 2x, then reinitialize the LayerManager.

    The last paragraph suggests that you have seen this issue many times?

  3. I saw you started to work on Exact Stack Rooting and you stopped to work on Lazy Bytecode Generation: why? Is it because the gain on GGC is higher or the work on LBcG is harder? Other reason? Thanks

  4. var sardonic = 0;
    var genuine = 1;

    I don’t have any good ideas on how to improve this situation.

    Sounds like a difficult situation assuming that future performance gains are as likely to come from GPU acceleration as anywhere else.

    Is it possible to take advantage of open source and investigate how Google and/or webkit address this issue and perhaps take some ideas from them?

    I guess I’m assuming they have more success in handling this issue than Mozilla does without having any real evidence of that. Maybe it’s an industry-wide problem and the only solution is to picket nVidia, AMD and Intel’s headquarters? :) Mr Torvalds and his finger might lend some support at nVidia :)

    • I suggest you pick up coding it yourself – there’s handy tutorials on the internet.

      You also could fix all the memory leaks and general slow performance you have too, if you learn it.

      Please let us know when your release will be so we can see the progress you make :)

      • What a strange comment. Not sure why you are adopting a personal approach to what was a genuine suggestion. I’ve never claimed that coding was easy and nor that I could do better than everyone else out there. On the contrary, I was genuinely acknowledging what seems to be a significant concern.

        AFAIK Firefox has benefited from adopting some techniques used in Chrome already and it is not unusual for browser vendors to co-operate on standards implementations so there is a factual basis for my suggestion. I can’t recollect the exact elements of Firefox that have been inspired or borrowed from Chrome though. Perhaps I’m mistaken.

        Please limit your comments to sensible, non-personal and potentially productive comments which is what I’m trying to do.

        • Which is what you are trying to do?

          Nothing you post is productive pd, its all the same “my Firefox is slow you guys need to fix it” in pretty much every Mozilla blog you frequent.

          Now what I suggested is a productive way for you to help with your slowness – fix Firefox yourself.

          Or perhaps its time for you to upgrade the Pentium 3 to something a bit faster or perhaps try a different browser, and save the walls of text you write in each article here.

          • I actually haven’t posted anything for a long time, certainly not any walls of text. Only a moron would claim that one single person could fix Firefox themselves so you want to get personal, there you go, take your own morose ‘walls of text’ and shove them.

            For your own information, I have a quad-core Xeon with 8GB of RAM running Windows 7 at work and that doesn’t stop Aurora from having occasional issues, which I expect, being Aurora. Part of the reason I haven’t posted anything of late is because Aurora has been a lot better. However even when I post positive experiences like that, some people can’t seem to grasp it. I guess ignorance is something some people can’t be cured of.

          • While I would once have agreed with your view that “pd just feels entitled and complains about anything and everything without doing anything useful,” pd has recently been admitting that these things are beyond his ability and may well be hard problems even for professional programmers, and even had a sane conversation with Taras Glek about providing profiler data.

            You may still be skeptical of pd and find some bugs, but it seems to me that, like firefox, pd is improving, and deserves to be re-evaluated with less regard for the past.

  5. Nicholas, a number of image related bugs are landing in Nightly 18. The temporary backouts notwithstanding, a blog post that talks to these changes will help the community fully appreciate the extent of these fixes.

    Thanks for these posts!
    Manoj