Automated Testing For Lightning!

When Standard8, Dmose and others on the Thunderbird team started using Mozmill for their test automation, we watched with envy and discussed how awesome it would be to do something similar for the Calendar Project.

But there were huge technical roadblocks in the way: could Mozmill automate the XBL-heavy calendar user interface? How reliable would any such automation be due to the way the views were drawn on the screen? How much time would it take someone to do it? None of us had the time or bandwidth to figure it out, so Philipp and I proposed it as a Google Summer of Code project and hoped that some brave soul would choose to volunteer for our project.

The brave soul that answered was Merike Sell, our Estonian localizer. She proved herself an incredibly skilled hacker, finding ways to make these tests work reliably on the Lightning UI across Windows, Linux, and Mac operating systems. She’s even created a complete shared module of calendar utility functions to make it much easier for anyone to add new tests in the future.

To give you an idea of what types of automatic testing she completed, we have the following tests checked in to calendar/test/mozmill.

  • Event Creation, Verification of Events in all Calendar Views
  • Event Dialog Input Testing, including UTF-8 Input testing
  • Task Dialog Input Testing
  • Task Pane View testing
  • Today Pane Testing
  • Recurring event testing for Daily, Weekly, Monthly, and Yearly recurrences
  • Recurring events are displayed properly in horizontal views
  • Extremely detailed Timezone tests to check for the most common timezone bugs

Underlying these tests are a basic set of APIs in the shared module testCalendarUtils.js. These APIs can help you do everything from fill in an event dialog to verify that an event is in the correct location on the screen in each view. Merike has written some excellent documentation for the API as well.

We’d like to thank both Frank Hecker and the Mozilla Foundation as well as David Ascher and Mozilla Messaging for making this possible. And we’d like to thank and congratulate Merike on a job well done. She braved through completely uncharted waters, found a dozen Mozmill bugs and created the first complete set of calendar automated tests in the history of our project.

Thanks Merike!

1 comment

  1. Well done, I really like this add-on.
    How do I tell Lightning to send a “Page” or SMS message?
    Thanks,
    Harold