Mozilla Add-ons Community Blog
Stats dashboard example
Categories: developers

Improvements to Statistics Processing on AMO

8 responses

We’re revamping the statistics we make available to add-on developers on addons.mozilla.org (AMO).

These stats are aggregated from add-on update logs and don’t include any personally identifiable user data. They give developers information about user adoption, general demographics, and other insights that might help them make changes and improvements.

The current system is costly to run, and glitches in the data have been a long-standing recurring issue. We are addressing these issues by changing the data source, which will improve reliability and reduce processing costs.

Usage Statistics

Until now, add-on usage statistics have been based on add-on updates. Firefox checks AMO daily for updates for add-ons that are hosted there (self-distributed add-ons generally check for updates on a server specified by the developer). The server logs for these update requests are aggregated and used to calculate the user counts shown on add-on pages on AMO. They also power a statistics dashboard for developers that breaks down the usage data by language, platform, application, etc.

Stats dashboard example

Stats dashboard showing new version adoption for uBlock Origin

In a few weeks, we will stop using the daily pings as the data source for usage statistics. The new statistics will be based on Firefox telemetry data. As with the current stats, all data is aggregated and no personally identifiable user data is shared with developers.

The data shown on AMO and shared with developers will be essentially the same, but the move to telemetry means that the numbers will change a little. Firefox users can opt out of sending telemetry data, and the way they are counted is different. Our current stats system counts distinct users by IP address, while telemetry uses a per-profile ID. For most add-ons you should expect usage totals to be lower, but usage trends and fluctuations should be nearly identical.

Telemetry data will enable us to show data for add-on versions that are not listed on AMO, so all developers will now be able to analyze their add-on usage stats, regardless of how the add-on is distributed. This also means some add-ons will have higher usage numbers, since the average will be calculated including both AMO-hosted and self-hosted versions.

Other changes that will happen due to this update:

  • The dashboards will only show data for enabled installs. There won’t be a breakdown of usage by add-on status anymore.
  • A breakdown of usage by country will be added.
  • Usage data for our current Firefox for Android browser (also known as Fennec) isn’t included. We’re working on adding data for our next mobile browser (Fenix), currently in development.
  • It won’t be possible to make your statistics dashboard publicly available anymore. Dashboards will only be accessible to add-on developers and admins, starting on June 11. If you are a member of a team that maintains an add-on and you need to access its stats dashboard, please ask your team to add you as an author in the Manage Authors & License page on AMO. The Listed property can be checked off so you don’t show up in the add-on’s public listing page.

We will begin gradually rolling out the new dashboard on June 11. During the rollout, a fraction of add-on dashboards will default to show the new data, but they will also have a link to access the old data. We expect to complete the rollout and discontinue the old dashboards on July 9. If you want to export any of your old stats, make sure you do it before then.

Download Statistics

We plan to make a similar overhaul to download statistics in the coming months. For now they will remain the same. You should expect an announcement around August, when we are closer to switching over to the new download data.

8 comments on “Improvements to Statistics Processing on AMO”

  1. rugk wrote on

    > It won’t be possible to make your statistics dashboard publicly available anymore.

    Wait, why?

    How is this related to the switch to telemetry-based stats collection?

    The intend of this feature was IMHO not to let coworkers/“members of a team” to see the stats, but anyone. Why should I hide such useful statistics if I can just expose them to the public in a transparent way? Don’t get this…

    1. Jorge Villalobos wrote on

      There are a few reasons that I didn’t get to in the post:
      (1) AMO doesn’t publicly disclose any data about self-distributed versions, and this setting would break that assumption. It’s a relatively minor issue since it’s the developer’s choice, but it would make the rollout more complicated because we would need everyone to again set their numbers to be public.
      (2) The stats dashboard is pretty much the only page that exists both in the privileged Developer Hub and the public-facing site, which use different technologies and have different designs. This has lead to the page having a very strange look that doesn’t match either, and it’s also harder to maintain.
      (3) This isn’t a very frequently-used feature. Developers that want to publish their detailed numbers can do it on their own. Of course it won’t be as easy as it is now, but all of the data is still available to developers.

  2. ash wrote on

    ‘Firefox users can opt out of sending telemetry data’ – from my experience most of the users (including myself) disable sending of telemetry data. In addition, sending telemetry data disabled for corporate users.
    Means statistics will be significantly off and meaningless.
    As an extension developer I can not support this change.

    1. Jorge Villalobos wrote on

      Automatic add-on updates can also be turned off by users, or for them. We have been comparing both data sets, and yes, as the post indicates the numbers from telemetry are generally lower. However, they track very well with the update numbers in terms of change through time and how languages, platforms, versions, etc. compare with one another, which is what we think is the most valuable aspect of the stats.

      1. Marat Tanalin wrote on

        Fwiw, for the two of my three addons, stats are now HALVED (311 vs. 571 for UsableHomeButton, and 125 instead of 263 for ConsistentHTTPS). That’s obviously much more than statistical spread. No idea how this can be useful.

  3. custom.firefox.lady wrote on

    Will the total number of users still be available on the public facing AMO page for the addon? If so, will the way it’s calculated change also?

    1. Jorge Villalobos wrote on

      Yes to both. The number that appears in the public page is calculated based on the last couple of weeks of usage stats, so that number will change but will still be available.

  4. Paulus wrote on

    telemetry disabled here. you just made your stats less relevant by sticking your nose where it does not belong. measuring bandwith on your end is legit. metering the user is not. sadly, Firefox has made itself irrelevant already in many other ways. look at the one metric that counts: web browser market share (measured at the server, not intrusive client side metrics)