Brief Outage for Phoenix Data Center Chassis

Sheeri

One of the chassis in the PHX1 datacenter was experiencing issues which took many services, including those on the generic web cluster offline and degraded others for approximately half an hour. Fixing the issue took approximately 15 minutes. Services should be back to normal.

For reference, the following web services were either downgraded, or unavailable:

generic cluster (contains many web apps)

bouncer
elasticsearch
etherpad
graphite
hangprocessor
input
input-celery
openshift
plugins and plugins memcached
puppetmaster
rabbit
socorro memcache

If you have any questions or concerns please address them to helpdesk@mozilla.com.