Mozilla DB News, Monday Jun 11th

Sheeri

We are as busy as ever at the Mozilla DB team. We are currently in the middle of conference season, and in the past two weeks I have spoken at OUG Harmony in Finland and at the SouthEast LinuxFest in Charlotte, NC. Here are the PDF slides from my presentations (video will be coming sometime later on):

What else have we been up to?

  • Added a dev category to our monitoring systems because we finally had a developer machine be too laggy in replication to use the standard monitoring check; the system was paging us, but it is a webdev machine so it was OK that it lagged.
  • Restored a backup of a dev database, because it was dropped and a dev needed it back. Remember that while dev may not be important to you, a dev machine is production to a developer. Similarly, a test/stage machine is production to a QA engineer.
  • Some network interface problems caused our internal databases to stop replicating.
  • Added some custom bugzilla fields and permission for the metrics user to access them.
  • Added permissions to our internal tools databases so we could failover gracefully
  • Dealt with more space issues from our puppet dashboard, as we mentioned last time. Our admin for that tool has gotten the space issues under control.
  • Turned off our temporary addons db servers, that we had spun up to deal with the 2 crashing servers we’d had.
  • Restarted MySQL on a developer machine to pick up the character-set-server=utf8 variable we’d set after manually converting tables that were set to latin1.
  • Continued backing up the small servers built in the data center move that had not yet been backed up. Most of these are not in heavy use or not important (but not all!)
  • Fixed a problem with postgres monitoring after our monitoring servers were moved – had to copy a .pgpass file on the monitoring machine.
  • Worked on restoring and de-duplicating entries in our instance of Bugzilla. Data duplication occurred when I had made assumptions about primary keys existing that did not, when I was doing a routine replication sync check. We ran into issues with NULL values when trying to de-duplicate, so the process was much more complicated than we thought it to be.
  • Created a new database and credentials for a javascript testnet, in dev.