This summer, I worked with the Mozilla Webdev team to create a new tool for addons.mozilla.org (AMO). The goal for my project was to create a framework that would scan add-ons uploaded to the site and perform a variety of tests on them. These could fall into a number of categories, and included simple well-formed tests, security tests, and tests specific to the type of add-on being uploaded.
The version that is currently live on AMO features many of the originally planned tests, along with several other new ones we added during the summer. The tests are run when a developer first uploads an add-on, and the developer can re-validate his/her add-on at any time. Most tests generate warnings that a developer might be doing something unsafe or unwanted, and serve as flags to encourage the developer to re-think his/her approach to the add-on. However, certain tests now prevent an add-on from being uploaded to AMO at all, giving both administrators and editors finer control over the content of the site. In addition to developers being able to review the results of the validation tests, editors can see the results when reviewing an add-on which is pending public nomination.
From a technical standpoint, the framework utilizes a variety of technologies to accomplish its tasks. Most tests are comprised of regular expression pattern matches, which match unwanted patterns in the extracted add-on files. Other tests check the add-on for blacklisted or whitelisted files, and ensure that these are missing or present as required. Some of the most interesting tests utilize a framework called jshydra, a static analysis tool for JavaScript. This tool allows us to analyze JavaScript in advanced ways that pattern matches simply cannot provide. For instance, one test checks for global variables in JavaScript files that could pollute the global namespace. Another test checks a specific file, and generates a warning if a function that is not on a whitelist is called. Though these are two basic tests using jshydra, the tool provides a huge amount of functionality that can be taken advantage of in the future.
