The new addons.mozilla.org (AMO) is up and running through its first peak load time without major issues. We will continue to monitor site performance today to verify that our performance adjustments indeed solved our load issues.
What did we do?
- Moved public files back to releases.mozilla.org, so those mirrors can do what they are good at — delivering files.
- Fixed image paths so they do not vary with locale. Since images are not localized, they do not have to have varying URLs. By limiting images to a unique URL, it improves cache rates.
- Adjusted caching rules. We had configured the load balancer to not cache all content when a user has logged in. Unfortunately that meant not caching images and other high-traffic services. We adjusted caching rules to force all images and service URLs to be cached regardless of whether or not a user is logged in.
Below are the load and traffic graphs as of 1040AM PST.
The load is much more linear, and at a lower level with our adjustments in place.
We have an increase in traffic, but this is expected as we have many more images and locales to support. The traffic spikes from last Friday were much higher than this, however, due to releases.mozilla.org traffic and lack of caching.
Overall I can’t say how happy I am to see some better results. Thanks to everyone who worked over the weekend with us to help troubleshoot. The battle isn’t over yet — we’re still looking for ways to tweak performance — but we can definitely see the light at the end of the tunnel.