JavaScript DOM

I filed bug 533874 to expose the JavaScript AST to JS, but turns out I need to explain why it’s important to expose the JavaScript AST. So lets start from the beginning.

My name is Taras and I like to wrestle useful information out of tools that do not think to offer it. I firmly believe that writing/maintaining code is more difficult than it should be. I think the current organization of compilers is partially responsible for it.

Related Prior Work
Some groups at Mozilla have realized that inspection and refactoring of code are tasks that will often consume as many people as can be thrown at it so there has to be a better way. That better way is through tools: get computers to do things that are difficult for humans. We do that a lot for other kinds of tasks, but not so much for computers. For example, when was the last something you googled something using telnet? Yet our tools for presenting and analyzing code are about as sophisticated as telnet. For some reason it is easier to find some random piece of information on the web than it is to find what code implements a function that is being called from my code.

As the original author, I think Dehydra, Treehydra on top of the GCC plugin API give us a pretty reasonable way to extract useful information out of our C++ code. I feel lost and confused without Dave’s DXR, the JS team routinely breaks (and fixes) invariants in the JS engine and we’ve been able to wipe out certain classes of bugs (ie code patterns that cause them) mozilla-wide. I’m looking forward to reducing the footprint of Mozilla by deleting more dead code.

JavaScript Needs
My efforts so far have focused on C++ because it is finicky language and developers need all the help they can get with it. I think we need to the same thing for JavaScript. I think there is a lot of useful information in JavaScript code that is burried and hard to get to. Joshua Cranmer broke new ground in building JSHydra on top of SpiderMonkey. This way one can parse JavaScript in exact same way as Spidermonkey sees it (something that can only be approximated with other approaches). I think we should offer the ability to analyze js code within Spidermonkey. Unfortunately, he is a busy student so he doesn’t have the time to develop JSHydra this to the next level.

How Would JavaScript Developers Benefit?

Currently we have jshydra deployed on AMO to scan javascript for potential issues to make the reviewer’s job more productive.
Dave Humphrey is working away on integrating JSHydra into DXR so we can have semantic navigation of JavaScript: what components are implemented JavaScript, wouldn’t it be nice to know who calls certain methods(including javascript callsites), etc. Gandalf has a cool idea where he needs to be able to extract translatable strings out of JetPacks. I believe JetPack people in general would like to be able to analyze JS code more.

I think it’s clear that the common trend in all these tasks is to turn an opaque text blob into something that can be easily navigated programatically. Wouldn’t it suck to not to be able to walk the DOM for HTML? Why is it ok to accept that handicap for the essense of all programs?

How Should This Be Solved?
I don’t know for sure, but my friend Jim Blandy, came up with a good suggestion. We already have eval/uneval in SpiderMonkey. it would be really cool to extend uneval to produce a iteratable data structure instead of a text blob. So I filed a bug 533874 on this, and this my explanation of why I think that would be a very empowering feature to expose ASTs in JavaScript.


  1. +1

    Make sure to coordinate with Brendan on this– he’s talked before about trying to create a set of interfaces for a structured representation of JS AST’s. Whatever you end up doing, it could set important precedents for the future of ECMAScript.

  2. Bernd der Bessere

    It’s welcomed to explain the abbreviations talked about.

    What means AST?

  3. AST means abstract syntax tree, it was discovered by the PHP Developers when they were writing PHP4. That’s how advanced it is. I am not surprised that JS devs would balk at providing the AST since it is at the CUTTING EDGE of programming language research.

  4. Of course I think more information about Javascript would be great! However, we need a runtime tool, not a static analysis tool. Javascript just isn’t static.