Download Statistics Update

In June, we announced that we were making changes to add-on usage statistics on addons.mozilla.org (AMO).  Now, we’re making a similar change to add-on download statistics. These statistics are aggregated from the AMO server logs, do not contain any personally identifiable information, and are only available to add-ons developers via the Developer Hub.

Just like with usage stats, the new download stats will be less expensive to process and will be based on Firefox telemetry data. As users can opt out of telemetry reporting, the new download numbers will be generally lower than those reported from the server logs. Additionally, the download numbers are based on new telemetry introduced in Firefox 80, so they will be lower at first and increase as users update their Firefox. As before, we will only count downloads originating from AMO.

The good news is that it’ll be easier now to track attribution for downloads. The old download stats were based on a custom src parameter in the URL. The new ones will break down sources with the more standard UTM parameters, making it easier to measure the effect of social media and other online campaigns.

Here’s a preview of what the new downloads dashboard will look like:

A screenshot of the updated statistics dashboard

We expect to turn on the new downloads data on October 8. Make sure to export your current download numbers if you’re interested in preserving them.

Extensions in Firefox 81

In Firefox 81, we have improved error messages for extension developers and updated user-facing notifications  to provide more information on how extensions are modifying their settings.

For developers, the menus.create API now provides more meaningful error messages when supplying invalid match or url patterns.  This updated message should make it easier for developers to quickly identify and fix the error. In addition, webNavigation.getAllFrames and webNavigation.getFrame will return a promise resolved with null in case the tab is discarded, which is how these APIs behave in Chrome.

For users, we’ve added a notification when an add-on is controlling the “Ask to save logins and passwords for websites” setting, using the privacy.services.passwordSavingEnabled settings API. Users can see this notification in their preferences or by navigating to about:preferences#privacy.

https://lh6.googleusercontent.com/Sy7Q3cLUc8lkcMvj6HI1hUKA7dM0YjkZnlHkFxVM3UeNjmGUzAeqbxTDlDdL2rCdZgKNa9KCkbioBvo_rQSHWkTcnSoAUIyxeBa4z7kkghffAvwfseVFopCmnJ1KX-ZF8FatwLSI

Thank you Deepika Karanji for improving the error messages, and our WebExtensions and security engineering teams for making these changes possible. We’re looking forward to seeing what is next for Firefox 82.

Introducing the Promoted Add-ons Pilot

Today, we are launching a pilot program to give developers a way to promote their add-ons on addons.mozilla.org (AMO). This pilot program, which will run between the end of September and the end of November 2020, aims to expand the number of add-ons we can review and verify as compliant with Mozilla policies, and provides developers with options for boosting their discoverability on AMO.

 

Building Upon Recommended Extensions

We strive to maintain a balance between openness for our development ecosystem and security and privacy for our users. Last summer, we launched a program called Recommended Extensions consisting of a relatively small number of editorially chosen add-ons that are regularly reviewed for policy compliance and prominently recommended on AMO and other Mozilla channels. All other add-ons display a caution label on their listing pages letting users know that we may not have reviewed these add-ons.

We would love to review all add-ons on AMO for policy compliance, but the cost would be prohibitive because they are performed by humans. Still, developers often tell us they would like to have their add-ons reviewed and featured on AMO, and some have indicated a willingness to pay for these services if we provide them.

Introducing Promoted Add-ons

To support these developers, we are adding a new program called Promoted Add-ons, where add-ons can be manually reviewed and featured on the AMO homepage for a fee. Offering these services as paid options will help us expand the number of add-ons that are verified and give developers more ways to gain users.

There will be two levels of paid services available:

  • “Verified” badging: Developers will have all new versions of their add-on reviewed for security and policy compliance. If the add-on passes, it will receive a Verified badge on AMO and in the Firefox Add-ons Manager (about:addons). The caution label will no longer appear on the add-on’s AMO listing page.

Add-on listing page example with verified badge

  • Sponsored placement on the AMO homepage. Developers of add-ons that have a Verified badge have the option to reach more users by paying an additional fee for placement in a new Sponsored section of the AMO homepage. The AMO homepage receives about two million unique visits per month.

AMO Homepage with Sponsored Ssection

During the pilot program, these services will be provided to a small number of participants without cost. More details will be provided to participants and the larger community about the program, including pricing, in the coming months.

Sign up for the Pilot Program

If you are interested in participating in this pilot program, click here to sign up. Please note that space will be limited based on the following criteria and restrictions:

  • Your add-on must be listed on addons.mozilla.org.
  • You (or your company) must be based in the United States, Canada, New Zealand, Australia, the United Kingdom, Malaysia, or Singapore, because once the pilot ends, we can only accept payment from these countries. (If you’re interested in participating but live outside these regions, please sign up to join the waitlist. We’re currently looking into how we can expand to more countries.)
  • Up to 12 add-ons will be accepted to the pilot program due to our current capacity for manual reviews. We will prioritize add-ons that are actively maintained and have an established user base.
  • Prior to receiving the Verified badge, a participating add-on will need to pass manual review. This may require some time commitment from developers to respond to potential review requests in a timely manner.
  • Add-ons in the Recommended Extensions program do not need to apply, because they already receive verification and discovery benefits.

We’ll begin notifying developers who are selected to participate in the program on September 16, 2020. We may expand the program in the future if interest grows, so the sign-up sheet will remain open if you would like to join the waitlist.

Next Steps

We expect Verified badges and homepage sponsorships for pilot participants to go live in early October. We’ll run the pilot for a few weeks to monitor its performance and communicate the next phase in November.

For developers who do not wish to participate in this program but are interested in more ways to support their add-ons, we plan to streamline the contribution experience later this year and explore features that make it easier for people to financially support the add-ons they use regularly. These features will be free to all add-on developers, and remain available whether or not the Promoted Add-ons pilot graduates.

We look forward to your participation, and hope you stay tuned for updates! If you have any questions about the program, please post them to our community forum.

Update on extension support in the new Firefox for Android

Last week, we finished rolling out the new Firefox for Android experience. This launch was the culmination of a year and a half of work rebuilding the mobile browser for Android from the ground up, replacing the previous application’s codebase with GeckoView—Mozilla’s new mobile browser engine—to create a fast, private, and customizable mobile browser. With GeckoView, our mobile development team can build and ship features much faster than before. The launch is a starting point for our new Android experience, and we’re excited to continue developing and refining features.

This means continuing to build support for add-ons. In order to get the new browser to users as soon as possible—which was necessary to iterate quickly on user feedback and limit resources needed to maintain two different Firefox for Android applications—we made some tough decisions about our minimum criteria for launch. We looked at add-on usage on Android, and made the decision to start by building support for add-ons in the Recommended Extensions program that were commonly installed by our mobile users. Enabling a small number of extensions in the initial rollout also enabled us to ensure a good first experience with add-ons in the new browser that are both mobile-friendly and security-reviewed.

More Recommended Extensions will be enabled on release in the coming weeks as they are tested and optimized. We are also working on enabling support for persistent loading of all extensions listed on addons.mozilla.org (AMO) on Firefox for Android Nightly. This should make it easier for mobile developers to test for compatibility, and for interested users to access add-ons that are not yet available on release. You can follow our progress by subscribing to this issue. We expect to have this enabled later this month.

Our plans for add-on support on release have not been solidified beyond what is outlined above. However, we are continuously working on increasing support, taking into account usage and feedback to ensure we are making the most of our available resources. We will post updates to this blog as plans solidify each quarter.

Disconnect’s road to success

Developers create extensions for a variety of reasons. Some are hobbyists who want to freely share their work with the world. Some find a way to turn their project into a small, independent business. Some companies build extensions as part of a business strategy. Earlier this year, we interviewed several add-on developers to learn more about the business models for their extensions. We learned a lot from those conversations, and have drawn on them to create upcoming experiments that we think will help developers succeed. We’ll be posting more information about participating in these experiments in the next few weeks.

In the meantime, we asked Disconnect CEO Casey Oppenheim to share his thoughts about what has made his company’s popular privacy-enhancing browser extension of the same name successful. Disconnect is an open-source extension that enables users to visualize and block third-party trackers. Together, Mozilla and Disconnect studied the performance benefits of blocking trackers and learned that tracking protection more than doubles page loading speeds. This work led us to build Enhanced Tracking Protection directly into Firefox in 2019 using Disconnect’s tracking protection list.

Today, Disconnect earns revenue by offering privacy apps at different price points and partnerships with organizations like Mozilla. They have also extensively experimented on monetizing the Disconnect browser extension to support its development and maintenance. Following are some of the learnings that Casey shared.

Why did you decide to create this feature as an extension?

Extensions are a really powerful way to improve user privacy. Extensions have the ability to “see” and block network requests on any and all webpages, which gave us the ability to show users exactly what companies were collecting data about their browsing and to stop the tracking. Browser extensions also were a great fit for the protection we offer, because they allow developers to set different rules for different pages. So for example, we can block Facebook tracking on websites Facebook doesn’t own, but allow Facebook tracking on facebook.com, so that we don’t break the user experience.

What has contributed to Disconnect’s success?

Our whole team is sincerely passionate about creating great privacy products. We make the products we want to use ourselves and fortunately that approach has resonated with a lot of users. That said, user feedback is very important to us and some of our most popular features were based on user suggestions. In terms of user growth, we rely a lot on word of mouth and press coverage rather than paid marketing. Being featured on addons.mozilla.org has given us great visibility and helped us reach a larger audience.

When did you decide to monetize your extension?

We began monetizing our extension in mid-2013, years before Firefox itself included tracker blocking. Since that time we have conducted several experiments that have always been based on voluntary payments, the extension has always been free to use.

Are there any tips you would want to share with developers about user acquisition or monetization?

We’ve learned a few lessons on this topic the hard way. Probably the most important is that it is very difficult to successfully monetize by interrupting the user flow. For example, we had the great idea of serving a notification inside the extension to try and get users to pay. The end result was terrible reviews and a bad user experience coupled with minimal increase in revenue. In our experience, trying to monetize in context (e.g., right after install) or passively (e.g., a button that is visible in the user interface) works better.

Is there anything else you would like to add?

Extensions are essential apps for billions of users. Developers should absolutely pursue monetization.

Thank you, Casey! 

Extensions in Firefox 80

Firefox 80 includes some minor improvements for developers using the downloads.download API:

  • When using the saveAs option, the save dialog now shows a more specific file type filter appropriate for the file type being saved.
  • Firefox now exposes internal errors in the Browser Console to aid debugging.

Special thanks goes to Harsh Arora and Dave for their contributions to the downloads API. This release was also made possible by a number of other folks from within Mozilla for diligent behind-the-scenes work to improve and maintain WebExtensions in Firefox.

Introducing a scalable add-ons blocklist

When we become aware of add-ons that go against user expectations or risk user privacy and security, we take steps to block them from running in Firefox using a mechanism called the add-ons blocklist. In Firefox 79, we revamped the blocklist to be more scalable in order to help keep users safe as the add-ons ecosystem continues to grow.

 

Cascading Bloom Filters

One of the constraints of the previous blocklist was that it required parsing of a large number of regular expressions. Each Firefox instance would need to check if any of its user’s installed add-ons matched any of the regular expressions in the blocklist. As the number of blocks increased, the overhead of loading and parsing the blocklist in Firefox became greater. In late 2019, we began looking into a more efficient way for Firefox to determine if an add-on is blocked.

After investigating potential solutions, we decided the new add-ons blocklist would be powered by a data structure created from cascading bloom filters, which provides an efficient way to store millions of records using minimal space.

Using a single bloom filter as a data structure would have carried a risk of false positives when checking if an add-on was blocked. To prevent this, the new add-on blocklist uses a set of filter layers to eliminate false-positives using the data from addons.mozilla.org (AMO) as a single source of truth.

The same underlying technology used here was first used for Certificate Revocation in CRLite. Adapting this approach for add-ons provided an important head-start for the blocklist project.

Further Optimizations: Stash lists

To reduce the need to ship entire blocklists each time blocks are added, an intermediate data structure is created to record new blocks. These “stash-lists” are queried first, before the main blocklist database is checked. Once the stash-lists grow to a certain size they are automatically folded into a newly generated cascading bloom filter database.

We are currently evaluating additional optimizations in order to further minimize the size of the blocklist for use on Fenix, the next major release of Firefox for Android.

Shipping this in Firefox Extended Support Release (ESR)

Firefox Extended Support Release (ESR) is a Firefox distribution that is focused on feature stability. It gets a major feature update about once per year and only critical security updates in between. When we first identified the need to move the blocklist to a cascading bloom filter in late 2019, we knew we had to land the new blocklist for ESR 78 or we would risk having to maintain two different blocklists in parallel until the next ESR cycle.

In order to land this feature in time for Firefox 78, which was slated to hit the Nightly pre-release channel in May, we needed to coordinate efforts between our add-ons server, add-ons frontend and Firefox Add-ons engineering teams, as well the teams in charge of hosting the blocklist and the still-in-development bloom filters library. We also needed to make sure this new solution cleared all security reviews and QA, as well as coordinate its rollout with Release Engineering, and make sure we had enough data to measure its success.

Our leadership encouraged us to land the new blocklist in Firefox 78 and ensured that we would get the cross-team support necessary to achieve it. Having all these hurdles cleared was very exciting and nerve-wracking at the same time, since now our main challenge was to deliver this huge project on time. While a late-breaking bug prevented us from landing the new blocklist in Firefox 78, we have been able to gradually roll it out with the Firefox 79 release and will enable it in an ESR 78 update.

This was an ambitious project, as it had many moving parts that required the support of many teams across Mozilla. During the project Crypto Engineering, Kinto, Security, Release Engineering, QA and data teams all made significant contributions to enable the Add-ons Engineering team to ship this feature in five months.

Thanks

None of this would have been possible without the help and support of the following people: Simon Bennetts, Shane Caraveo, Alexandru Cornestean, Shell Escalante, Luca Greco, Mark Goodwin, Joe Hildebrand, JC Jones, Dana Keeler, Gijs Kruitbosch, Thyla van der Merwe, Alexandra Moga, Mathieu Pillard, Victor Porof, Amy Tsay, Ryan VanderMeulen, Dan Veditz, Julien Vehent, Eric Smyth, Jorge Villalobos, Andreas Wagner, Andrew Williamson and Rob Wu.

Openness and security: a balancing act for the add-ons ecosystem

Add-ons offer a powerful way for people to customize their web experience in Firefox. From content blocking and media enhancement to productivity tooling, add-ons allow third-party developers to create, remix, and share new products and experiences for the web. The same extensibility that allows developers to create utility and delight in Firefox, however, can also be used by malicious actors to harvest and sell user data.

With an ecosystem of 20,000+ extensions hosted on addons.mozilla.org (AMO), hundreds of thousands of self-distributed extensions, and millions of users around the world, finding the right balance between openness and security is a key challenge for our small team. Developers need to feel supported on our platform, and users need to feel safe installing add-ons, so we continually make adjustments to balance these interests.

Adapting our review model

Prior to the adoption of a new extensions API in 2017, buggy or malicious add-ons could take nearly full control of Firefox, and in some cases, a user’s device. Because these extensions could do so much potential damage, all add-ons hosted on addons.mozilla.org (AMO) had to pass human review before they could be released to users. This led to long delays where developers sometimes waited weeks, if not months, for their submissions to be reviewed. In some cases, developers waited months for an add-on to be reviewed, only to have it rejected.

The transition to the new extensions API greatly limited the potential for add-ons to cause damage. Reducing the attack surface enabled us to move to a post-submission review model, where extensions undergo automated checks and are prioritized for human review based on certain risk factors before becoming available, usually within a few hours. All add-ons are subject to human review at any time after publication.

Human reviews are still necessary

Since the transition to a post-submission review model, we have continued to make adjustments to our products, systems, and processes to maintain a balance between user safety and developer support. While we’ve made gains in new mechanisms to combat malicious activity, human review remains the most reliable method for verifying the safety of an add-on because of the complex and contextual nature of add-on code written in JavaScript.

However, human code review is a resource-intensive activity. As we weighed our options for how to keep add-ons safe for users in 2019, it became clear that we only possessed the resources to guarantee human reviews for a small number of extensions. Because we already had an editorial program in place for identifying and featuring add-ons, it made sense to build a trusted add-on program off past curatorial efforts. This became the Recommended Extensions program.

Currently, we human-review every version of each of our 100+ Recommended Extensions before publication. Beyond that, our limited review resources are focused on monitoring and stamping out malicious activity that may be lurking in our ecosystem. For a sense of scale, AMO receives 20,000+ new version submissions per month.

Since we can only guarantee human-review for all versions of Recommended Extensions, AMO applies a warning message to the listing pages of all non-Recommended extensions. The intention of this message is to let users know that since a non-Recommended extension may not have been reviewed by a human, we can’t guarantee it’s safe.

Developer feedback and future plans

We’ve heard feedback from developers whose add-ons are not in the Recommended program that they are concerned the warning message can discourage users from installing their add-ons. Some have asked whether it’s possible to request human reviews for their add-ons so they can be badged as safe to install. We are exploring ways to better support these developers and provide more discovery opportunities for them.

During the remainder of 2020, we will experiment with new programs to address these issues and help more extensions become successful. Please stay tuned to this blog for updates on the upcoming experiments and opportunities for participation, and head to our community forum with any questions or feedback.

Extensions in Firefox 79

We have a little more news this release: a new API method, a reminder about a recently announced change, a preview of some things to come, and a few interesting improvements. Let’s get started!

Warming up tabs

To optimize resource usage, render information on inactive tabs is discarded. When Firefox anticipates that a tab will be activated, the tab is “warmed up”. Switching to it then feels much more instantaneous. With the new tabs.warmup function, tab manager extensions will be able to benefit from the same perceived performance improvements. Note this API does not work on discarded tabs and does not need to be called immediately prior to switching tabs. It is merely a performance improvement when the tab switch can be anticipated, such as when hovering over a button that when clicked would switch to the tab.

Changes to storage.sync

We’ve blogged about this recently, but given this is part of Firefox 79 I wanted to make sure to remind you about the storage.sync changes we’ve been working on. Storage quotas for the storage.sync API are now being enforced as part of backend changes we’ve introduced for better scalability and performance.

There is no immediate action required if you don’t use the storage.sync API or are only storing small amounts of data. We encourage you to make your code resilient while your storage needs grow by checking for quota errors. Also, if you are getting support requests from users related to stored preferences you may want to keep this change in mind and support them in filing a bug as necessary.

For more information and how to file a bug in case you come across issues with this change, please see the blog post.

Firefox site isolation coming later this year

The Firefox platform team has been working on a new security architecture that isolates sites from each other, down to separating cross-origin iframes from the tab’s process. This new model, nicknamed Fission, is currently available for opt-in testing in Nightly. The platform team is planning to begin roll-out to Nightly and Beta users later this year.

So far, we have identified two changes with Fission enabled that will impact extensions:

  • Content scripts injecting extension iframes (from a moz-extension:// url) and accessing them directly via the contentWindow property will be incompatible with Fission, since that iframe will run in a different process. The recommended pattern, as always, is to use postMessage and extension messaging instead.
  • The synchronous canvas drawWindow API will be deprecated, since it’s unable to draw out-of-process iframes. You should switch to the captureTab method, which we are looking to extend with more functionality to provide a sufficient replacement.

If you are the developer of an extension that uses one of these features, we recommend that you update your extension in the coming months to avoid potential breakages.

We’re working to make the transition to Fission as smooth as possible for users and extension developers, so we need your help: please test your extensions with Fission enabled, and report any issues on Bugzilla as blocking the fission-webext meta bug. If you need help or have any questions, come find us on our community forum or Matrix.

We will continue to monitor changes that will require add-ons to be updated. We encourage you to subscribe to our blog to stay up to date on the latest developments. If more changes to add-ons are necessary we will reach out to developers individually or announce the changes here.

Miscellaneous

  • Extensions can use webRequest listeners to observe their own requests initiated by the downloads API.
  • The tabs.duplicate API now makes the tab active before resolving the promise, for parity with Chrome.
  • Disabling and re-enabling a WebExtension which provides a default search engine now correctly sets the engine as default again.

Special thanks in this release goes to community members Myeongjun Go, Sonia Singla, Deepika Karanji, Harsh Arora, and my friends at Mozilla that have put a lot of effort into making Firefox 79 successful. Also a special thanks to the Fission team for supporting us through the changes to the extension architecture. Stay tuned for next time!

Changes to storage.sync in Firefox 79

Firefox 79, which will be released on July 28, includes changes to the storage.sync area. Items that extensions store in this area are automatically synced to all devices signed in to the same Firefox Account, similar to how Firefox Sync handles bookmarks and passwords. The storage.sync area has been ported to a new Rust-based implementation, allowing extension storage to share the same infrastructure and backend used by Firefox Sync.

Extension data that had been stored locally in existing profiles will automatically migrate the first time an installed extension tries to access storage.sync data in Firefox 79. After the migration, the data will be stored locally in a new storage-sync2.sqlite file in the profile directory.

If you are the developer of an extension that syncs extension storage, you should be aware that the new implementation now enforces client-side quota limits. This means that:

  • You can make a call using storage.sync.GetBytesInUse to estimate how much data your extension is storing locally over the limit.
  • If your extension previously stored data above quota limits, all that data will be migrated and available to your extension, and will be synced. However, attempting to add new data will fail.
  • If your extension tries to store data above quota limits, the storage.sync API call will raise an error. However, the extension should still successfully retrieve existing data.

We encourage you to use the Firefox Beta channel to test all extension features that use the storage.sync API to see how they behave if the client-side storage quota is exceeded before Firefox 79 is released. If you notice any regressions, please check your about:config preferences to ensure that webextensions.storage.sync.kinto is set to false and then file a bug. We do not recommend flipping this preference to true as doing so may result in data loss.

If your users report that their extension data does not sync after they upgrade to Firefox 79, please also file a bug. This is likely related to the storage.sync data migration.

Please let us know if there are any questions on our developer community forum.