Snappy #36

Misc

I have been using dates to mark passage of time in the Snappy project. I think I’ll switch to a simple counter instead. We are ~36 updates into this project.

Blogging

Ludovic Hirlimann blogged about how spotlight spends a lot of time indexing the Firefox network ‘Cache’ directory (known problem, bug 718910). If you experience this problem and would like to see it fixed, please comment in the bug if the suggested remedy helps.

Tim Taubert wrote about reducing new-tab jank. I mention Tim a lot in these updates. He takes on a lot of interesting bugs in Firefox frontend. Hopefully he’ll make a habit out of blogging about his work.

Networking

Nick Hurley landed a change to reduce our maximum cache size to 350 megabytes. In order to avoid excessive disk IO traffic old cache size of 1 gigabyte remains in effect until the cache is reset. See bug 709297 and Nick’s blog post for more details. Progress is also being made on bug 777328 so we can move towards not blowing away our cache 10-20% of the time.

Michal Novotny is proceeding with incrementally reducing cache-caused jank that’s due to holding a lock on the main thread while doing IO on a background thread. He also removing a multitude of synchronous necko APIs, see bug 695399.

Patrick McManus is removing synchronous proxy-related code, see bug 766973 for the DNS-related bit. Turns out our proxy code also does all kinds of synchronous operations when detecting proxy configuration, etc. This is being worked on, but hasn’t been filed yet.

I usually try to highlight work that has already landed, but in this case it is important to point out that the Necko team is working hard on addressing significant problems in the networking code. These problems are tricky and will take a while to fix. The team is relatively new and is still discovering hidden surprises in their codebase.

Profiler

Benoit Girard posted a preview of view-source in the profiler. This will be handy for figuring out where performance problems lay, especially in JS files that have been preprocessed (our JS preprocessor does not try to keep the line numbers sane).

14 comments

  1. If the cache is now limited to 350MB, does that mean that HTML5 games need to go out of their way to store assets in IndexedDB to get fast load times?

  2. If you want fast, you want to use indexeddb filehandle stuff. I’d have to see some numbers before I would generalize that 350mb is not enough for html5 games. You can mod Nick’s extension and do some experiments.

  3. What I understood from the cache problem is that searching the harddisk is taking too long with 350+MB caches. This would mean that SSD’s wouldn’t have this problem right? In a few years most people will have SSD’s so will there be benchmarks with it?

  4. Well, the concern is not that 350MB isn’t enough for a single game – it probably is in many cases, given fast enough network to stream more content in on demand – but it seems like that means that only a few games will ever be able to stay retained in cache at any given time. If so, that’s really unfortunate.

    IDB + FileHandle might be enough to solve this, but it would suck if existing games suddenly became worse experiences.

    I’ll try to spend some time messing with Nick’s extension.

  5. I think the cache doesn’t currently fully respect the user’s pref for size limit.

    I use a 128M ramdrive for cache (I don’t mind it being deleted on reboot), and if running long enough, even 90M limit pref results in the ramdrive running out of space. I reduced the limit gradually, and ended up with about 70M limit pref, which still didn’t run out of space.

    I think the limit doesn’t take the index files into account, and they can be many MBs.

    This is probably worth a bug, but I’m posting it here anyway since this post mentions cache size limit.

  6. Avih, the cache takes account of contents of the index files. But those files are allocated in chunks, so it probably does not take account various overhead…
    Please file

  7. 350MB cache, seriously? Because it’s working so well for chrome: watching a youtube video effectively clears your cache of all the useful items.
    Even with the ~gig of FF cache right now, 2/3 of that is video crap that has no good reason for being there.
    Why is video content even first written to %TEMP% for flash / mozilla-media-cache for webm and then again to the real cache?

  8. Hey Taras, I was wondering if you knew about this tool:

    http://gameangst.com/?p=320

    (Along with his accompanying blog posts about reducing exe bloat/compile times)

  9. In the nightly options –> setting the cache manually is still suggested to be 1024mb. Shouldn’t this also be 350mb?

  10. Something unrelated to this news: for some days I get real bad hangs, sometimes lasting seconds, mainly while changing tabs. I also tried with a new profile. The gecko profiler always shows something like this:
    http://people.mozilla.com/~bgirard/cleopatra/?report=542efa04d60977a067a3623a00765d837cb952ba

  11. @kamulos,
    thanks I think that’s bug https://bugzilla.mozilla.org/show_bug.cgi?id=783748

    Really appreciate the report.

  12. @kumalos

    Your profile helped us identify a new issue:
    https://bugzilla.mozilla.org/show_bug.cgi?id=784756

    thanks :)

  13. I am glad to help. The profiler really is an awesome tool.