Mozilla Scheduled Downtime – 08/21/2008, 7pm – 10pm PDT (0200 – 0600 08/22/2008 UTC)

We will have a scheduled maintenance window tonight from 7pm to 11:00pm PDT. This is being tracked in master bug 451495. The following changes will take place:

  • 7:00pm PDT (0200 UTC) DNS change. As part of our CDN trial/test, we’ll be offloading to CDNetworks and pushing a DNS change to make that active. See bug 450744 for details. No downtime is expected.
  • 7:00pm PDT (0200 UTC) updates. We’ll be updating to pick up code updates. No downtime is expected.
  • 7:00pm PDT (0200 UTC) duration 4 hours: Kernel upgrades. We’ll be doing kernel security updates on various machines. The full list of affected machines and services is listed below.
  • 8:00pm PDT (0300) UTC) database & app server upgrades. Both the databases and app servers will be getting memory upgrades to improve performance. Duration 30 minutes.

Tonight’s kernel upgrades will affect the following services for approximately 10 minutes each:

  • All database backed websites, including:
    many others

  • All sites that use LDAP for authentication
  • Internal DNS services in the MPT colo
  • SVN, Hg
  • MXR
  • half of (gravel users will be failed over to sand)
  • mozilla-europe content management
  • External geodns nameservice
  • web interface for, and mailing lists
  • search services for
  • build graphs (old and new)
  • pageload test server
  • tinderbox symbol upload
  • try build server
  • breakpad

Please let me know if you have any reason why we should not proceed with this planned maintenance. As always, we aim to keep downtime to as little as possible, but unexpected complications can arise causing longer downtime periods than expected. All systems should be operational by the end of the maintenance window.

As mentioned, this maintenance window is tracked in bug 451495. Feel free to comment directly in that bug (or this blog) if you see issues past the planned downtime.

Update: breakpad will not be included in tonight’s batch of kernel updates.