In February 2008 I wasn’t even a user of Lightning yet as it becomes obvious from looking at my home calendar. Four months later I was researching how to translate it so I could have Thunderbird and Lightning in the same language because an Estonian localization didn’t exist.
A year later being familiar with the localization side of Mozilla and using Lightning regularly the option to write tests for calendar looked really appealing when I noticed it in the wiki. In April I found myself participating in the testday for Mozmill 1.1. (There’s another one coming where you could participate by the way!)
Now that I’ve written smoketests for bug #500469 I find Mozmill relatively similar to Selenium as I expected. Both of these tools test user interface functionality and therefore the basic three types of steps are common to both:
- Locating elements to act on
- Specifying actions to be done
- Checking the results
Element location in a Mozilla application can be done either by using Mozmill’s inspector which gives you a ready to use locator or with the help of DOM inspector if you need to access an element that is further down in the hierarchy of UI elements than what Mozmill’s inspector gives you. Locators in Lightning are quite long but mostly relatively easily accessible. There are exceptions to this though: bug #505336 being an example of an element that is fighting against automation.
As UI responsiveness is always slower than script speed pauses between test actions are essential to let UI load properly. Failing to do that can result in an otherwise fine-looking test not working and once in a while I’m still able to forget that despite of knowing it. There are also some inconsistencies between platforms support of Mozmill which made it impossible to stick to the Linux environment I mostly use but it also gives both the Mozmill and Calendar development version stress on both platforms which can only make these more bulletproof.
Result checking is the most important part of functional tests even if running a test without any asserts can reveal some of the bugs too. This is also the part of Mozmill code that recently reminded me of just how quickly Mozmill is developed. I was looking for a way to assert that a certain element is not hidden and found a assertPropertyNotExist function by scrolling through the available methods on Google Code. This seemed like a good way to make sure that there’s no hidden attribute attached to an element. To my surprise the first call to this method caused a failure which stated that there’s no such function. It had been added only a couple of hours earlier and the version installed didn’t have it yet!
Active development of Mozmill and fantastic people on both the #qa and #calendar IRC channels make the difference between a person merely checking things out and a new participant in the project. That’s what it takes to get a new person to participate.