IT goings-on

For those curious what IT has been up to lately, wonder no longer – here’s a quick status update from the past week, highlighting just some of the great stuff the Mozilla IT team has been working on recently.

First up, our team is growing:

  • Welcome Chris Knowles, who will be officially helping us out as a Storage and Virtualisation admin (unofficially, he’s going to help us with an even greater challenge: safely landing our Kerbals on the Mun).
  • A big hello to the entire OpSec team, who are now part of the IT team proper.  As separate entities we already had a close and fruitful working relationship – now that we’re all together, things are only going to get better!  Expect more updates about all the interesting stuff we’ll be working on together – as long as it’s not classified top secret. 😉

Our fabulous SRE team had an epic bug squashing session during which they reduced their open bugs by over 40%.  Given how diverse their queue is at any given moment, this is quite the accomplishment – great work guys!

Speaking of SRE’s, our very own Dumitru Gherman, along with developer Emma Irwin, made the long trip from Mountain View out to London to host a session on “Hacking your online safety” at MozFest.

Local Windows machine deployments at our San Francisco office used to take hours, but thanks to the efforts of Mike Poessy at the SFO service desk, via a combination of templating and new imaging techniques, this has been reduced down to as little as 20 minutes.  This newly streamlined process will almost certainly find its way to our other offices as well.

And last, but most certainly not least, the Release Operations team has been very busy:

  • Jake Watkins wrote a module for Windows called “metric-collective” that polls system stats and forwards the results to graphite/statsd.  This is going to be rolled out to all the Windows build and test systems during the next release cycle.  Combined with related initiative to roll out collectd across all of the OS X and Linux machines, we’re going to have a whole new level of graphing and trending available across the entire release infrastructure.
  • Mark Cornmesser and Q Fortier have been hard at work on a brand new imaging and management mechanism for our Windows 2008 build hosts.  The biggest change is a move away from our old, manually maintained monolithic image, to a modular image complete with proper change and configuration management.  This new platform is currently being tested on a number of project branches and is expected to be rolled out to mozilla-central, try, and inbound in the next few weeks.
  • Dustin Mitchell worked with the Auto Tools team to set up an independent Puppet instance, itself destined to automate the management of the entire QA Mozmill CI infrastructure.  He also set up local Python and NPM mirrors for use by Mozillians, providing a compelling model and implementation for other use-cases going forward.

As always, if you have any questions or want to learn more, feel free to comment below or hop on to #it on  See you next time!