Download Counts Halted

Mike Morgan

6

The download controller was modified on Thursday to prepare for the release of the 1.5.0.12 -> 2.0.0.4 major update. During release cycles, AMO takes abnormally high load which sometimes causes interruptions in service.

To avoid this situation we agreed to cache public download hits from the AMO install buttons. This does two things:

  • Relieve application load by allowing the hardware load balancer to cache file requests — which are ultimately redirects to releases.mozilla.org
  • Relieve the database by not having constant inserts on the download table — which causes extra load because of indexes that were put on the table to make count updates work correctly

The benefits can be seen below. First, traffic for a web node:

traffic for AMO

Second, a CPU usage graph from the same node:

cpu usage

You can see the positive effect on load near noon on Wed.

The status of download counts is tracked in bug 384084. The plan is to create a background maintenance script that parses AMO logs via a cron job in off-peak hours. Counts will be updated once a day.

In addition to this, we are taking the opportunity to do a couple of things. For our summer goals, we plan on improving statistics for developers by offering:

  • Actual update ping counts in bug 384086
  • An improved API for aggregating add-on statistics and integrating it into your blog or external sites

We plan on resuming download counts early next week and update pings should be available by mid July. Thanks for your patience as we make adjustments to accomodate server load, and have a great 4th of July!

Categories: AMO

6 responses

  1. Frédéric Wenzel wrote on ::

    Unless I am mistaken, the effect is actually visible at 12:00 or noon rather than midnight on Wednesday.

    Other than that, good article :)

  2. morgamic wrote on ::

    Err, yeah you’re right — fixed that. :)

  3. Chuck Baker wrote on ::

    “We plan on resuming download counts early next week and update pings should be available by mid July.”

    No download counts yet … Are they coming back soon? I really miss tracking my extension popularity.

  4. Milos wrote on :

    According to AMO I had 2 downloads this week. :) When will this feature be fixed?

  5. Wil Clouser wrote on ::

    Hey,

    Sorry for the delays. The download count script progress is being tracked here:

    https://bugzilla.mozilla.org/show_bug.cgi?id=384084

  6. lwd wrote on :

    Is the download count back to work? Why I still don’t see any update on the download count? The download of my extension is still 1:(