Mozilla DB news, Week of Fri Mar 9th

This week I am in Mountain View to help with some physical data center moves. It is nice to get to see where the magic happens, and our new data center is pretty awesome. Our data center technicians, Derek Moore and Erica Muxlow, are really doing a stellar job, as are their interns.

At the office:
– Visited the San Francisco office. What a great view!
– A dependent subquery that had been running fine for weeks started acting up, causing 1000% more load than normal on a machine that hosted many sites, including the Firefox Flicks $10,000 Contest website and Tinderbox Push LoadLog [whoops, Freudian slip!]. We changed it to a join and added an appropriate index, and the query now runs much faster and is not causing any load.
– Created a user and database for the development site of Mozilla Labs
– Pointed our webdev team to some newer exports for some data they were looking for
– Enabled MySQL access for another buildbot host
– Cleaned up some older binary logs that were hanging around after a binlog name change in January, also set expire-logs-days for the same server
– Converted some data from latin1 to utf8 on a staging machine and debug’d a cron job that was duplicating data
– Helped assemble 32 new SSD drives for new database blades (and thanks to the rest of the systems team for installing the drives and kickstarting the machines!)

At the data center:
– Racked 16 blades, including scanning in and updating the inventory database with serial numbers and asset tags
– For 6 of those 16, had to physically dig out the toe tags that had the serial numbers on it (due to a manufacturing problem I’m sure)
– Attached rails to several new servers