MemShrink progress, week 25

It was a good week for MemShrink.

ObjShrink

The biggest news this week is that Brian Hackett landed his objshrink work.  This has roughly halved the size of Firefox’s representation of JavaScript objects.  It’s also reduced the amount of memory Firefox uses for shapes, a related data structure.  If  you look at about:memory you can see that “gc-heap/objects” and “gc-heap/shapes” entries account for many MBs of memory, so this is a big win.  Read here (under the “Objects” and “Shape” headings) if you want more details.

Memory Reporting

I added some more JavaScript memory reporters and fixed some problems with the existing ones.  Specifically…

  • New: “runtime/threads/temporary” counts various pieces of short-lived data, mostly parse nodes.  It sometimes spikes up to multiple MBs.
  • New: “runtime/threads/normal” measures thread data on the heap that isn’t covered by “runtime/threads/temporary”.  It’s normally a few 100s of KBs.
  • New: “runtime/contexts” measures JSContexts and various related structures, which usually account for a few 10s or 100s of KB.
  • New/fixed: “runtime/threads/regexp-code” measures code generated by the regexp compiler.  This was previously measured under “mjit-code/regexp” but that reporter was broken recently when the location of this generated code was moved, so I fixed it.
  • Fixed: “runtime/atoms-table” and “runtime/runtime-object” are existing reporters that together often account for up to 4.5MB.  They were incorrectly marked as measuring non-heap memory instead of heap memory, meaning that the “explicit” and “heap-unclassified” totals were both incorrectly inflated by that amount.  While fixing this, I also confirmed that we weren’t making the same mistake with any of our other memory reporters.
  • Renamed: “runtime/threads/stack-committed” was previously called “stack-size”.  I renamed it to go with the other “runtime/threads” reports because that’s where it conceptually belongs.

I also added a memory reporter for XPConnect.  It typically accounts for 1MB or more.

I have a lot more work in the pipeline to improve the coverage and correctness of memory reporters.  More about this in the coming weeks as I land the relevant patches!

Other

Joel Maher got RSS memory measurements working for Android on Talos, Mozilla’s performance regression testing suite.  This was a MemShrink:P1 bug.  Graphs of the live data can be seen here.

Benoit Jacob made some improvements how Firefox manages and reports memory used by WebGL.

Bug Counts

Here’s the current bug count.

  • P1: 27 (-1/+1)
  • P2: 141 (-3/+5)
  • P3: 62 (-1/+2)
  • Unprioritized: 0 (-0/+0)

Only three more MemShrink bugs than this time last week!  Pretty good.

19 Responses to MemShrink progress, week 25

  1. I honestly dont feel / see any memory different with ObjectShrink. May be i was expecting too much. It doesn’t even reduce the JS memory by 10%.

    So my guess is that 50% reduction are in the best case scenario. Which doesn’t happen as often.

  2. @Ed: It was 50% of objects, not total js engine usage. On #developers:
    bhackett: 5% of total, 9% of js
    bhackett: there will be another bug in the next two weeks which should take another 2% off the browser’s total usage

    • Yes i do realize it was for Object and Shape, but given both of them represent up to 50% of JS Memory, a 50% reduction would have meant a 25% reduction in JS memory. Which wasn’t the case.

      Let’s hope IGC and other Snappy bugs wont disappoint.

      • Nicholas Nethercote

        Your numbers are bogus. First, all the “gc-heap/objects” and “gc-heap/shapes” numbers don’t get anywhere near 50% of JS memory usage. Second, while “gc-heap/objects” has roughly halved, “gc-heap/shapes” shrank by a much smaller amount.

  3. Well yes, the wins from ObjShrink are not as noticeable as I had hoped.
    On the http://lights.elliegoulding.com/ demo however, the compartment goes from 70M to 50M (on x64) so that is a clear win there.

    Another question as I see you have mentioned runtime/threads/stack-committed: Wasn’t there a plan to cut this down a bit on linux? It is constantly using 8M which is quite a bit of memory.

    • Nicholas Nethercote

      It’s 8MB of address space, but it’s mostly not touched so doesn’t really cost anything.

  4. latest nightly crashes after being idle for a few minutes. caused by those MemShrink changes?

    last 2 crashes:

    https://crash-stats.mozilla.com/report/index/bp-1a8d91c5-4360-47ea-8248-effb22111207

    https://crash-stats.mozilla.com/report/index/bp-163656ca-1d5e-4a49-98ff-417a72111207

    I never had such permanent crashes before.

    André

  5. When I access about:cache?device=disk, heap-unclassified goes to 44%. This isn’t a memory leak (if I close the tab, in some seconds heap-classified go back to 25%), but I think there’s something on about:cache not covered by any memory reporters.

    • Nicholas Nethercote

      I tried that and it stayed around 21% for me with a Nightly build. What version of Firefox are you running?

      • Firefox 10.0 a2, Aurora. I tested with build 2011-12-07.

        My internet is slow (1Mbit) , and because of that I use a big amount of cache.

        Number of entries: 100417
        Maximum storage size: 4000000 KiB
        Storage in use: 3397931 KiB

        • Nicholas Nethercote

          Oh, right. So the page is enormous, right? It’s probably covered by one of the existing CSS or DOM memory reporter bugs. I’m working on it :)

  6. You won’t see Objectshrink improvements in Nightly yet, because, to my knowledge, the patches are only in the jaegermonkey branch. No jm -> m-c merge has happened yet either.

    Correct me If I’m mistaken.

  7. Just a question. (I know its not directly memory related) : how do i know wich tab in FF is causing most CPU activity… ?

    Details :
    i have 30+ tabs opened on my aurora. memory used by FF is pretty stable BUT my cpu activity keeps going from some percents to 30-50% at 2 or 3 seconds rate. Is it normal ? If no, how do i find which tab is causing this ?

    Side note : Thx a lot for your memshrink work. I discovered “restore_on_demand” option yesterday, and i find it really handy. Perhaps should be more publicized.

    • Nicholas Nethercote

      There’s no easy way to find out CPU usage per tab, unfortunately. The 30–50% spikes don’t sound normal, but it depends on what sites you are visiting.

    • run a xperf trace with profile stackwalking and look what causes the high CPU usage at the spike.

  8. As a more casual user than most who follow this blog meaning I have no technical training, I think memshrink is finally showing something tangible to me. The readings in about:memory in Chrome finally shows that Firefox nightly is a lot more memory efficient than Chrome in regards to my browsing habits which is having on average less than 10 tabs open at once.

    I started to pay attention to memory usage b/c my computer has less than 2 gigs of ram and Firefox 4 and subsequent version had locked up my computer more than once b/c of memory issues.

    While every personal experience is unique and may not indicative at all, I just wanted you to know that as a user I appreciate memshrinks efforts. I am sure other users will feel the same.