Benchmarks and Instrumentation for Fennec/etc

I wrote Fennecmark to automate some of the tasks that I did manually while doing performance debugging.

I tried to capture some of the “perceived performance” in numbers. My goal is to focus on user-visible areas of performance. Ideally it will enable us to track performance better to ensure that key features do not regress in performance and enable us to compare Fennec speed on various platforms. I need to spend some quality time with QA people to figure out how to achieve that.

Currently Fennecmark loads a slow-to-load webpage, zooms around it and then pans from the top to the bottom. This measures: responsiveness during pageload, zoom speed and panning lag.

See code at http://hg.mozilla.org/users/tglek_mozilla.com/fennecmark.

JSD Instrumentation

Spidermonkey provides an API that allows one to get a notification on every method entry/exit. I was able to do most of my Fennec performance analysis via a component in bug 470116. My stopwatch component times the execution of every js function call and spits out a log that has been very useful in figuring out what is taking up time in Fennec chrome.

Porkstain

I am itching to write a tool that can instrument large portions of Mozilla code such that it can be profiled across C++/JS boundaries and without any external tool support. I am guessing this would be most useful on platforms with crappy sampling tools, but it would be cool if it made finding slow codepaths easier in general. If you know any lightweight instrumentation techniques, please share.

I wrote a little prototype to insert stopwatch stuff into code deemed interesting by oprofile (stuff in the bug above). The code patching part works well, but it’s a big runtime hit and outputs too much data.

3 comments

  1. See “Baxter, Ira. Branch Coverage for Arbitrary Languages Made Easy” at http://www.semanticdesigns.com/Company/Publications/ for a discussion on how to instrument arbitrary languages using a program transformation tool. The relevance is that a program transformation tool like DMS can transform C++ source code (it has a robust C++ front end). Given that you are working on Pork (I’m unclear on how this is different than Oink?) that uses Elsa, I would have thought the ideas in this paper would have been, er, obvious.

    Of course, you can also get a C++ profiler from us, that works using exactly this technique.

    Ira Baxter, CTO Semantic Designs

  2. Ira,
    Your company’s stuff is pretty interesting, hope your clients realize how awesome this stuff is. The paper doesn’t really mention any non-obvious instrumentation stuff, but thanks anyways.

  3. I want to do this analysis for Windows Fennec. Can you tell me how to go ahead. I saw the patches but they are for linux build.