Mozilla IT & Operations
While all of the SCL3 work discussed in this blog goes on, we are still growing in other data centers as well. As I mentioned last week we have a lot of gear going to Phoenix. This week, 3 network engineers, 2 sysadmins, and 1 data center project manager descended on our Phoenix facility to… Read more
Mozilla is growing, and we have to grow our infrastructure right along with it. A lot of this just happens behind the scenes as we are buying and installing new gear every month. For today’s post I thought I would give an insight into the broad range of servers we have in the ordering/shipping process… Read more
During a recent Mozilla all-hands event Laura Thomson held a short presentation, titled “Working with IT”. Laura was the right person to give it and the feedback that we have gathered is that we need to help people understand how to work with IT, and help you all understand how our infrastructure works. Expect more… Read more
Recently we saw instability in one of our admin nodes. The hardware is a bit old so it is easy to throw that out as a cause. This is an HP DL360, 4th Generation (current is g7 with g8 right around the corner). Yet, blaming the hardware should be a last resort. Server problems are… Read more
Here at Mozilla, we have grown to thousands of servers in a short period of time (and even more individual instances when you count virtual machines). Like most other organizations, we have to rely on tools that help the sysadmins keep their sanity at such scale. We have picked Puppet as our tool of choice,… Read more
We will have a scheduled maintenance window tonight from 4:00pm to 10:00pm PST. The following changes will take place: * 4:00pm PST (0000 UTC) addons.mozilla.org – database update, bug 621486. This is a minor update and any outage here should be brief. * 4:00pm PST (0000 UTC) input.mozilla.org – code update, bug 621565. This is… Read more
