MemShrink progress, week 93–94

After lots of activity in the previous month, the past two weeks have been fairly quiet for MemShrink.

AWSY

areweslimyet.com has been proving its worth.

Jonathan Kew reduced the amount of memory taken by fonts on Fennec at start-up, which was detected by AWSY/mobile.  Jonathan also reverted a change that AWSY detected as increasing Fennec memory consumption, and filed a follow-up to investigate further.

Joe Drew fixed a bad regression on AWSY relating to image decoding.  It’s not clear to me if this was a genuine regression that users would have seen, or if it was an artifact of the way AWSY does its measurements.  Either way, it’s good that it was fixed, and props to Joe for doing it so quickly.

Finally, we closed bug 833518, which was for an AWSY regression caused by the new DOM bindings.  This was previously improved by an Aurora-only hack, but enough cases have been translated to the new bindings that we’re naturally down almost to where we were.

Miscellaneous

The mobile team abandoned their goal of making Fennec work on phones with only 256 MiB of memory.  The rationale is that Android phones with only 256 MiB of RAM are uncommon, whereas low-end phones that meet the current minimum of 384 MiB are much more common.  The mobile team will of course continue to look for ways to improve memory consumption in order to make life for users with 384 MiB phones.

I modified the JS engine so that it doesn’t emit bytecode for asm.js functions in the normal case.  This reduced the memory consumption of the Unreal 3 demo used at GDC by about 100 MiB.  I also added a memory reporter for array buffers used by asm.js, which are often quite large and weren’t being measured on 64-bit platforms.

Alexandre Poirot fixed a leak relating to dev tools.

Randell Jesup fixed a small leak in WebRTC.

Help Needed

I’m working on adding a button to about:memory trigger the dumping of memory reporter data to file.  I have a patch awaiting review, but I’m getting a test failure on Windows.  The test saves gzipped memory reports to file, and then immediately loads that saved file (and uncompresses it) and checks the data looks as expected.  This works fine on Mac and Linux, but on Windows I’m sometimes getting incomplete data in the load step.  The file is quite short (just 253 bytes compressed, and 620 bytes uncompressed) and the truncation point varies between runs;  in the most severe occurrence only 9 bytes of uncompressed data were loaded, though the cut-off point seems to vary randomly.

I suspect there’s a file synchronization problem between the save and the load, even though gzclose() is called on the save file before the loading occurs.  If anyone has ideas about what the problem might be, I’d love to hear them.

Update: Nils Maier and an anonymous commenter pointed out the problem — I was using “r” instead of “rb” for the file mode.  On Windows, this causes mangling of EOL chars.

Also, we’re seeing some strange behaviour on Mac OS X where memory managed by jemalloc doesn’t appear to be released back to the OS as it should.  This is both alarming and hard to understand, which is not a good combination.

Good First Bugs

I have two easy bugs assigned to me that I probably won’t get around to for some time.  Both of them would be good first (or second, or third…) bugs.

  • Bug 857382 is about making about:memory handle memory report diffs more elegantly.
  • Bug 798914 is just a minor code clean-up.  Nothing too exciting, but first bugs often aren’t!

Please email or comment in one of the bugs if you are interested in helping.

Bug Counts

Here are the current bug counts.

  • P1: 15 (-0/+2)
  • P2: 138 (-4/+8)
  • P3: 129 (-2/+7)
  • Unprioritized: 1 (-3/+0)

7 Responses to MemShrink progress, week 93–94

  1. Robert O'Callahan

    Is the file read being cut off at a Ctrl-Z character in the compressed file data? That used to be treated as EOF in some DOS/Windows I/O APIs.

    • Nicholas Nethercote

      Good question! The contents of the file changes slightly each time because it includes the PID, so that might explain why it’s different each time. You’d think that gzread() surely can handle this… but I’ll take a look. Is the ctrl-Z char 26 (“sub”) in ASCII?

  2. Robert O'Callahan

    Yes.

  3. good job, as always, but that unreal demo is available for public?

    • emscripten.org says “Unreal Engine 3 ported to JavaScript+WebGL in just 4 days.” “(live link to demo will hopefully be available soon).”

  4. + nsresult rv = aFile->OpenANSIFileDesc("r", &file);

    The file open mode needs to be binary, i.e. “rb” not “r”. Windows is the only platform where there is a distinction between binary and text mode for file IO.