The download controller was modified on Thursday to prepare for the release of the 18.104.22.168 -> 22.214.171.124 major update. During release cycles, AMO takes abnormally high load which sometimes causes interruptions in service.
To avoid this situation we agreed to cache public download hits from the AMO install buttons. This does two things:
- Relieve application load by allowing the hardware load balancer to cache file requests — which are ultimately redirects to releases.mozilla.org
- Relieve the database by not having constant inserts on the download table — which causes extra load because of indexes that were put on the table to make count updates work correctly
The benefits can be seen below. First, traffic for a web node:
Second, a CPU usage graph from the same node:
You can see the positive effect on load near noon on Wed.
The status of download counts is tracked in bug 384084. The plan is to create a background maintenance script that parses AMO logs via a cron job in off-peak hours. Counts will be updated once a day.
In addition to this, we are taking the opportunity to do a couple of things. For our summer goals, we plan on improving statistics for developers by offering:
- Actual update ping counts in bug 384086
- An improved API for aggregating add-on statistics and integrating it into your blog or external sites
We plan on resuming download counts early next week and update pings should be available by mid July. Thanks for your patience as we make adjustments to accomodate server load, and have a great 4th of July!