May 26th, 2011 by jseward
This is really a RFC.
It seems to me that many end-user bugzilla reports about excessive memory use share a common structure. First, the user reports that “I did A, B and C, had a coffee, went back to my machine and found that Firefox was sitting on N gigabytes of memory.” Then follows lots of
discussion along the lines of “oh, so the cycle collector did/didn’t run while Fx was idle”, and “but no, that’s not right, because the tertiary FooBar timer inhibit mechanism will have made the XYZ collector run instead” kind of thing.
This goes on and on, while everybody tries to figure out what Firefox was actually doing in the period leading up to the too-much-memory observation. Meanwhile the original reporter gets bored with the discussion and moves on to something else, and there’s general confusion, annoyance, and lack of reproducibility all round.
So the idea is simple: make a log file listing events which are known to have a significant impact on memory usage. Nothing heavyweight, just one line per event, timestamped, plus brief relevant stats. For example:
- GC started / ended, total size N, reclaimed M bytes
- GC mapped in new heap / unmapped heap
- CC started / ended, total size N, reclaimed M bytes
- image discard ran
- jemalloc mmap’d more pages / munmapped some pages
- nanojit / mjit mapped / unmapped code pages
Perhaps with some indirectly relevant events such as
- downloaded another chunk of the phishing database
- no user input seen for 17 minutes
- new tab created; now there are 23 of them
- user requested about:memory (+ what it produced)
- extension Xyzzy loaded/initialised
That way, we’d at least have some information about the space history leading up to a situation where a user says “urk! massive memory leak!”.
The log would be low overhead, so it can be used in production. For sure we don’t want to log more than a couple of events per second. Perhaps a moderate sized circular buffer (64KB? 1MB?) that could be dumped to disk on request. Then, when someone reports excessive memory use, the first thing we ask for is the log file.
10 Responses to “A low overhead, always-on, system-wide memory event log”