We’ll be taking advantage of the end-of-holiday lull and post-Fennec 1.0rc1 / pre-Firefox 3.6 releases to perform maintenance on some Production & Release Engineering infrastructure.
During this window, all check-in trees will be closed.
We’ll be performing maintenance Sunday from 12:00pm to 6:00pm PST . The following changes will take place:
- 12:00pm PST (2000 GMT) Duration 2 hours: NetApp OS upgrade. We’ll be updating the two Network Appliance file servers that handle a number of Release Engineering VMs. See bug 531233 for more details.
- 12:00pm PST (2000 GMT) Duration 2 hours:
b01database cluster maintenance. We’ll be upgrading the firmware on the
b01database cluster (bug 535859) and performing maintenance to clear up disk space (bug 528573). This work will affect the following services:
- 1:00pm PST (2100 GMT) Duration 4 hours: ESX hardware upgrade. We’ll be upgrading the production ESX cluster from old dual CPU HP DL385 servers to our standard 8-core/32GB RAM HP BL460 platform. Since this upgrade involves switching CPU platforms (AMD to Intel), it requires each VM to be shutdown, migrated and powered up. Each VM’s downtime will be not unlike a normal reboot.The following list includes notable VMs & services that will be briefly affected:
Please let me know if you have any reason why we should not proceed with this planned maintenance. As always, we aim to keep downtime to as little as possible, but unexpected complications can arise causing longer downtime periods than expected. All systems should be operational by the end of the maintenance window.
Feel free to comment directly if you see issues past the planned downtime.