Give DXR a try

At Mozilla we have a long history of using MXR for looking up and discussing source code. Unfortunately MXR is an unlovable mess of Perl and a crappy (in terms of performance and license) text indexing engine that is glimpse. It is dead because nobody wants to work on it.

DXR is a semantically-aware successor to MXR. Semantic information is extracted from LLVM during compilation. This makes it possible to do searches like derived:nsIFile. DXR uses a modern Full Text Search engine for text searches, so it should be much faster than MXR. There is a test instance at dxr.mozilla.org, please give it a try. The homepage lists sample searches you can do.

DXR is written in Python. It uses an SQLite database + FTS index as a backend. Useful semantic information is extracted from the source via a Clang LLVM plugin. Checkout the source code at github.

DXR should be getting close to feature parity with MXR. Give it a try and let me know of any bugs/missing features you encounter (or submit a patch!). I realize that people have gotten used to various MXR quirks and that it can be stressful to switch to a new code indexer while trying to get stuff done, but MXR IS DEAD. We need to move on. Mozilla is complex, finding relevant code can take quite a while, especially for new contributors. Using a smarter indexer should save time, reduce frustration and free up a few developer-years to make Firefox better.

We have lots of ideas for DXR, but first we need to ensure it is a suitable replacement for MXR. Take DXR for a spin!

14 comments

  1. Justin Wood (Callek)

    One current issue, is when I (for example) search for a known filename: (e.g. devicemanagerSUT.py) I get taken directly to it.

    HOWEVER there is no idication of where in the tree I am, and no way to “browse” files from there.

  2. Justin Wood (Callek)

    Also during browsing I should not see

    http://dxr.lanedo.com/mozilla-central/–GENERATED–/

    + browsing file-tree pages has a broken ;-)

  3. Justin Wood (Callek)

    Lastly, any file not in hg shouldn’t appear in the index/search.

    Most notable example is:
    http://dxr.lanedo.com/mozilla-central/configure.html

  4. Justin Wood (Callek)

    O and we’ll need a solution for comm-central and other trees currently indexed in MXR that are different from m-c itself.

    [I think thats all my comments for now]

  5. Source code highlighting would be swell. Makes reading some of the longish files a tad easier.

  6. dxr gives random
    “problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.”

    I haven’t figured out how to see a particular version of a
    file using dxr. mxr supports that with
    ?rev=

    Also, how does one do marking using dxr?

    Every time I’ve tried dxr, I’ve noticed some basic
    functionality missing.

  7. Both are useful. For me, mxr has better search, and dxr has context-sensitive links. Which is why I wrote this quick and dirty bookmarklet which switches between mxr/dxr for the file you’re currently browsing. Not bullet proof, but still worked for me whenever the file exists on both systems: http://pastebin.mozilla.org/1648163

  8. Robert O'Callahan

    How do you search for the file names matching some pattern in DXR? E.g. MXR has http://mxr.mozilla.org/mozilla-central/find?string=nscoord. I can’t figure out how to do that with DXR. Searching for “path:nscoord” gives “Error: unknown search parameters”.

  9. Please don’t disrespect Perl. Like most languages – computer or physical – it can be used or abused. Too many write it like programming shorthand and then there’s the simple lack of readability some developers can’t help but write with. That’s as much the developer’s fault as anything to do with the language itself.

  10. Can you fix the dxr.mozilla.org -> dxr.lanedo.com redirect? I want to use dxr.mozilla.org for links in long lived places so those links will continue to work after DXR is moved to mozilla.org, but currently links with query strings break. Specifically:

    http://dxr.mozilla.org/search.cgi?tree=mozilla-central&derived=nsIFrame%3A%3ADidSetStyleContext

    breaks because “search.cgi” is dropped during the redirect.

    Also, I’m not sure if this is expected right now, but that search currently fails to produce any results even if the URL is fixed up manually.

  11. It is probably also a problem that the redirect causes the query string to be double escaped, as in the “…mozilla-central&derived…” becomes “…mozilla-central&derived…”.

  12. The blog software ate that. The & becomes &

  13. DXR is currently also missing Windows support (your nsIFile example doesn’t show nsLocalFileWin); MXR is just dumb enough to be able to do it ;)

    It’s also more visually complicated (more styling), though I guess userContent.css / stylish should be able to take care of that.

    Just a note to other readers: the lanedo.com stuff is because they’re working on DXR; that’s the instance we should be using.

    For my personal use, I find that I just ignore everything smart and use substring searches, often with RE filtering, on MXR. DXR’s text searching seems a bit weak, though I haven’t tried it recently.

  14. Mook,
    I’m not sure what MXR can do in nsIFile case that DXR can’t. You can always rerun a dumb text search in DXR, or there some limitation I’m not grasping?

    roc, that’s a bug. Everybody else, thanks for your comments, these are really helpful and will help us get DXR closer to production quality.