Jun 09

DXR: The Most Impressive Code Navigation Tool Ever

Are you are a developer who has been frustrated by the pathetic state of the art in code search engines and code browsing experience on the web? Have you been longing for being able to view code with more aid than coloured words?

David Humphrey just released his DXR bombshell. The “basic” concept behind DXR is extraction of the rich semantic information gathered by tools like GCC, Spidermonkey and xpidl. This data is then coherently linked together into a pretty UI in order to provide cleverness during code browsing sessions.

DXR will be happy to answer seemingly trivial queries:

  • List implementations of interfaces in C++ (and soon JS).
  • Provide relevant search results by searching semantic data first. No, grep is no longer state of the art for searching code.
  • Switch between definition and declaration.
  • Walk up/down class hierarchies.
  • Lookup typedefs, types, etc.

I’ve been wanting to see this sort of tool built on top information exposed by Dehydra since I got it working. Words can not express how pumped I am about DXR and the magic powers that we will be granting it.

Jun 09

Studying Fennec Performance

Working on Fennec performance N810 has been very educational. I have been learning more and more about performance profiling on crappy platforms. I define a platform as crap if it has poor development tools, limited OS or other significant limitations. Linux is a crappy platform in this case because it’s running on ARM where oprofile barely does anything and there are no other performance tools for N810 (more modern ARM cpus should user-space perf counters and be more useful for instrumentation).

JSD Secret Sauce for JS Optimization

In general there is a misconception that implementing stuff in JavaScript will result in slower code than doing the same in C++. That may be true if the code is implemented in the exact same manner. But in real life the expressiveness and safety of a high-level language like JavaScript permits algorithmic optimizations that would often not be realistic to do in C++ because of time/safety constraints.

So how does one figure out what’s slow in JS? timeless suggested that I checkout the JavaScript Debugging API. Using the API and a small hack in spidermonkey(JSD doesn’t expose fast dom calls) I was able to hook into chrome code to get a timed trace of JavaScript functions being run.

Once I had a trace it was relatively easy to figure out to “do not call this slow thing all the time” dance (aka optimize code). I collected that work in bug 470116. Last I checked there was relatively little room for optimization left on the JS side of Fennec, so then I went to look at what’s lurking in C++.

PS. Firebug is a popular consumer of the JSD API for those times when one isn’t willing to write JS components to figure out why something is slow.

C++ Is Harder

I’ve some success with inserting probes into C++ code. I would find interesting code by running oprofile on the desktop (while doing things that I felt were slow on N810). Oprofile would then provide me a callgraph which I would visualize with this awesome little script. Then I would stick MeasurerOfTime timing blocks into interesting “hot” code and hope that I would learn something useful.

This got me thinking. Wouldn’t it be nice if there existed a JSD for C++? It’d be cool to inspect the C++ callgraph just like one does for JS. It seems like it would help on platforms that aren’t gcc and can’t inject tracing code via -finstrument-functions. Even -finstrument-functions is of limited use due to the pain of looking up symbols in shared libraries. Stay tuned.

Measuring Progress

The worst part of doing optimizations is knowing that some time in a future an innocent programmer will slightly change some seemingly innocent code and things will no longer happen quickly. Short of policing every single patch by people who previously optimized code in question there is only one thing one can do: performance tests.

Fennecmark is a benchmark for measuring responsiveness of the Fennec features that I worked on most: panning, zooming and lag during pageload. I blogged about it before. Since then Joel Maher has gotten Fennecmark to run automatically and produce results on the graph server. I think we should be logging more numbers (Tpan, Tzoom), but it’s an excellent first step in monitoring performance regressions.

Jun 09

Dehydra & Pork Sources Moved

I moved dehydra to a more official location, please update your scripts and hg settings.
New dehydra url:

Pork got reshuffled during the move, it’s now 2 repositories. oink is dead. It now depends on current versions of flex (as opposed to flex-old) and features a cleaned up buildsystem.

New way to checkout pork:

hg clone http://hg.mozilla.org/rewriting-and-analysis/pork
hg clone http://hg.mozilla.org/rewriting-and-analysis/elsa pork/elsa

Jun 09

Dehydra/Treehydra C support

Every once in a while people want to be analyze C code. If you are one of those people, checkout bug 494960. For GCC 4.3, you’ll need to update your patch queue. If you you followed the Dehydra install instructions do something like:

cd gcc-4.3.0/.hg/patches
hg pull -u
cd ../..
hg qpush -a

and rebuild gcc to get C support in the plugin framework for GCC 4.5 (patch for trunk is in the bug).

Everything other than process_function should work in both treehydra and dehydra. Note that the objects provided by GCC will be slightly different in structure due to the underlying difference between gcc/g++, so scripts that were written for C++ may need to be updated.