Web QA: 2015 – Year in Review

(Sorry it’s late – better late than never, yeah?)

As we did last year, it’s always great to revisit and highlight all the awesome work we did over the course of a year.

A lot of the great progress we made last year sets us up with really strong groundwork, in both technical and collaborative aspects, for an equally (if not — and hopefully moreso!) rewarding 2016!

Pictured, above, from left to right (clockwise): Rebecca, Matt, Stephen, Krupa, Dave, Justin, and Bob.

Full-size images here: https://www.flickr.com/photos/134774698@N08/23715685901/in/album-72157662496585105/

Team Highlights:

• announced in June, but starting his contributions back in January, even, we welcomed Dave Hunt to the team, officially!
• Stephen stepped into a new, individual-contributor role, and is happy to continue working with this great team, in a new capacity 🙂
• to help bring our own team together a little more, as well as make our roles and services/offerings a little clearer to those with whom we work, we discussed and put our mission statement up on our team wiki
• we identified the need for a “Buildmaster” role to better-identity, track, and drive build failures to resolution, which we instituted, and use across the entire team, on a rotation model

• this also falls under Community, but, by and large, one of our biggest and most-impactful undertakings of 2015 was to finally make our Jenkins instance public.  There’s a much-more detailed blog post about it here: https://blog.mozilla.org/webqa/2015/03/05/web-qa-publishes-automated-test-results/
• through a combination of manual testing and our Selenium WebDriver automation, we tested and helped ensure that more than 15 website or Web applications were fully functional, end-to-end
• to test and isolate upcoming Selenium releases, and help ensure our Grid was more reliable, we set up a staging Selenium Grid instance
• working closely with WebOps on HTTP/2, we helped test early support for HTTP/2 in an upgrade to our Virtual Traffic Manager.

• Summer: coordinated, tested, and help ship both in-product and accompanying Web pieces for the June 1st launch of the Firefox Growth Campaign
• Fall: all of the above efforts, again, for the November 3rd launch of the Firefox Growth Campaign

• we collaborated on a new README for our test-automation projects – AMO’s README was the 1st to use
• we standardized on Flake8, and implemented automatic pull-request checks, run via Travis CI
• New py.test plugins, for which there’s a really great blog post, here http://blargon7.com/2015/12/selenium-tests-with-pytest/ :
  • pytest-selenium
  • pytest-html
  • pytest-variables

• through both manual testing and our test automation, we supported a Bouncer/Sentry rewrite and migration to AWS (both huge endeavors, to be sure)

• we started off the year by prototyping automated functional tests using The Intern, in largely-asynchronous JavaScript (as opposed to our usual Python)
• we helped test and ensure a smooth launch of the v1.0 of BrowserCompat

• we got our real-browser, full-stack Marketplace test automation running, via Travis CI, for the fireplace repo
• support (including test automation) for Mobile Optimized Websites
• tested and shipped Desktop games
• tested and shipped two rounds of navigation refreshes, across all platforms
• Firefox OS 2.5 features tested and launched:
  • Add-ons
  • Late Customization
  • Homescreens
  • Blocklisting

• Florin resurrected test-automation coverage, to help ensure that key, new profile features and supporting changes were tested, all throughout a crucial development cycle

• while our development team migrated the site/app (and its staging server) to the cloud, we cleaned-up and bolstered our test automation (just a sampling of the commits over the year)
• we also reinvigorated the conversation around application quality through better workflow and deployment practices

• we returned to, and renewed our efforts in both automated and targeted manual testing to SUMO:
  • we resurrected some high-level mobile tests
  • supported new features and revisited where we could help with additional test coverage

• we began automated functional tests for the Treeherder UI

• we introduced and welcomed our shiny-new webqabot to our IRC channel!

• We started recording and archiving our weekly team meetings via Air Mozilla
• Conferences:
  • Stephen attended Selenium Conference 2015 and published a summarizing writeup
• Presentations and Meetups:
  • The team held a meetup, during which Dave Hunt presented on our pytest-based test-automation framework
    • slides
    • video