Gzipped JSON is now the preferred format for attaching memory report data to bugs

Old: Text

about:memory is our main memory profiling tool.  You can copy and paste the memory reports shown by about:memory and they will reproduce nicely — so long as the displaying text is a fixed-width font — as the following example shows.

Main Process

Explicit Allocations
536.22 MB (100.0%) -- explicit
├──217.45 MB (40.55%) -- window-objects
│  ├───91.55 MB (17.07%) -- top(https://mail.google.com/mail/u/0/#label/A-moz%2Fbugz/13d41da77907a707, id=16)/active
│  │   ├──61.18 MB (11.41%) -- window(https://mail.google.com/_/mail-static/_/js/main/m_i,t/rt=h/ver=sobKNxkH_hk.en./sv=1/am=!rsy2VLKefbT1RPF-0_JbQoI4nDfwRxdk-CbEiLu_TGxDGzw05OfUOUODZR_O2flp0CcdPg/d=1)
│  │   │  ├──56.29 MB (10.50%) -- js
│  │   │  │  ├──56.11 MB (10.46%) -- compartment(https://mail.google.com/_/mail-static/_/js/main/m_i,t/rt=h/ver=sobKNxkH_hk.en./sv=1/am=!rsy2VLKefbT1RPF-0_JbQoI4nDfwRxdk-CbEiLu_TGxDGzw05OfUOUODZR_O2flp0CcdPg/d=1)
│  │   │  │  │  ├──35.23 MB (06.57%) -- gc-heap
│  │   │  │  │  │  ├──18.17 MB (03.39%) -- objects
│  │   │  │  │  │  │  ├───9.88 MB (01.84%) ── dense-array
│  │   │  │  │  │  │  ├───6.08 MB (01.13%) ── ordinary
│  │   │  │  │  │  │  └───2.21 MB (00.41%) ++ (2 tiny)
│  │   │  │  │  │  ├──10.71 MB (02.00%) ++ (6 tiny)
│  │   │  │  │  │  └───6.35 MB (01.18%) ── unused-gc-things
│  │   │  │  │  ├──12.01 MB (02.24%) ++ (7 tiny)
│  │   │  │  │  └───8.86 MB (01.65%) ++ objects-extra
│  │   │  │  └───0.19 MB (00.04%) ++ compartment(https://mail.google.com/_/mail-static/_/js/main/m_i,t/rt=h/ver=sobKNxkH_hk.en./sv=1/am=!rsy2VLKefbT1RPF-0_JbQoI4nDfwRxdk-CbEiLu_TGxDGzw05OfUOUODZR_O2flp0CcdPg/d=1, about:blank)
│  │   │  └───4.89 MB (00.91%) ++ (4 tiny)

Accordingly, until now the preferred way of reporting about:memory’s data in bug reports has been to include a copy of the entire contents of about:memory?verbose.

This worked well when about:memory contained a few dozen lines.  But the level of detail in the underlying memory reporters has increased greatly since then.  This is a good thing — more data is helpful — but it means that a full about:memory?verbose is now typically thousands of lines.  When viewing the data directly in about:memory?verbose this isn’t a problem, because you can easily collapse and expand sub-trees to focus on the interesting parts.  However, when reading pasted data as text, it’s overwhelming, even for experts.

New: Gzipped JSON

With the landing of bug 848560, about:memory can now import and export memory reports as gzipped JSON.  Use the “Write reports to a file” and “Read reports from a file” buttons at the bottom of about:memory.  (Note: the UI will likely change soon, but not substantively.)

(The JSON schema is documented in a comment near the top of xpcom/base/nsIMemoryInfoDumper.idl, for those who are interested.)

This is now the preferred format for attaching memory report data to bug reports.  It has two important advantages over text.

  • When you load one of these files into about:memory, you can do the aforementioned collapsing and expanding of sub-trees.
  • If you have two of these files, you can diff them using toolkit/components/aboutmemory/tools/diff-memory-reports.js.  (For the moment you’ll have to ungzip both files first, which is stupid.  Hopefully this’ll be fixed soon.)

The ability to compute memory report diffs will be extremely useful for diagnosing cases where Firefox’s memory consumption increases unexpectedly.  (There are already two bugs that are waiting on this feature for a better diagnosis.)

There is a bug open to implement a “Load reports from URL” feature in about:memory, which will make the viewing memory reports attached to bugs even easier.  It’s currently unassigned;  please email me or comment in the bug if you are interested in implementing it.

Finally, although the gzipped JSON format is now the preferred format for transferring the full memory report data, copy and paste is still useful for communicating snippets of about:memory’s contents.  Support for it won’t be disappearing.

5 Responses to Gzipped JSON is now the preferred format for attaching memory report data to bugs

  1. I hope that the JSON file is smaller in size as well than a copy-paste.
    I took a about:memory?verbose dump in bug 859747 . It was 2.3 MB in size! So i had to compress it and then upload, potentially making the bug less accessible.

    • The json file about:memory is producing is gzipped (compressed); and being text with large numbers of repetitive items should compress heavily.

    • Mine is 80KB :D

    • Nicholas Nethercote

      From my current session:

      about:memory?verbose cut and paste: 321,979 B
      gzipped JSON: 80,327 B
      ungzipped JSON: 1,861,935 B

      That’s exactly why we’re gzipping them.

  2. WindmillTilter

    I’ve always thought it would be good to output the about:memory information as XML. It’s a standard format, and each node is collapsible, and could have attributes for size, percent, url, etc.

    The example above would look something like:

    <![CDATA[

    ]]>

    Well, my example didn’t come through very well, even surrounded in a CDATA section, but you get the idea.

    Users could view it in their favorite XML reader, and code for reading/writing it is ubiquitous nowadays.

    Thanks !