WebExtensions in Firefox 47

We last updated you on our progress with WebExtensions when Firefox 46 landed in Developer Edition (Aurora), and today we have an update for Firefox 47, which landed in Developer Edition last week.

While WebExtensions will remain in a beta state in Firefox 47, we’ve made lots of progress, with 81 bugs closed since the last update. As of this update, we are still on track for a milestone release in Firefox 48 when it hits Developer Edition.

There’s a new way for you to get involved! Tell us which APIs you’d like support for by filling out this survey, to help us better prioritize them. We have also created a wiki page filled with resources to support developers through all the changes coming in the add-ons world.

APIs Implemented

Adding keyboard shortcuts that trigger actions in your extension arrived with the partial implementation of commands. This allows developers to map key presses from the manifest to actions in your add-on.

With the partial implementation of downloads you can download files from your add-on and get updates of the download progress. You can also search through the existing downloads using the search API.

Several additions and changes in webRequest have been included in Firefox 47. These prelude even bigger news for add-ons that focus on security and privacy to work perform the sort of network inspection needed to do their job. More details on these changes are covered in Giorgio’s blog post.

Also completed is the i18n API and we are getting closer to completing the bookmarks API.

More of the tabs API have been completed, including:

Several improvements have been made to the windows API, including :

  • Added support for creating a new window from an existing tab, popup-type windows
  • Querying and changing a windows’ minimized, maximized and fullscreen state.

All asynchronous methods which accept a callback function will now return a Promise if no callback is passed. These promises resolve at the same time a callback would normally be called, and reject with the value of lastError in cases where that would otherwise be set. Additionally, onMessage listeners may now return a Promise object if they wish to send a reply. When the promise resolves, its resolution value will be returned to the sender.

WebExtension manifests now support a “creator” property, which is displayed in the Add-on Manager, to indicate the author of the add-on. Additionally, manifests are now fully type-checked at startup, and any type errors or unexpected properties are reported to the Browser Console for inspection.

For the full list of API changes, please see the bug list.

Command line tool

A new command line tool is being build for WebExtensions, called “web-ext” will allow you to run, test and sign add-ons easily from the command line. For example to run your add-on in a new profile, from your add-on, just call: web-ext run.

This command line utility is in its early days, but you can follow along with web-ext development on github.

New examples

We’ve been preparing examples of our APIs on github. In the last few weeks we’ve added two more examples:

  • bookmark-it: an add-on that toggles a bookmark for the currently active tab.
  • tabs-tabs-tabs: an add-on that demonstrates some of the tabs APIs available (currently move, duplicate, reload and remove)

There are currently ten example add-ons available in the repository. All of them can be installed easily by cloning the repository locally and then installing temporarily through about:debugging.
Over the coming months we will work our way towards a beta in Firefox 47 and the first stable release in Firefox 48. If you’d like to jump in to help, or get your API added, please join us on our mailing list or at one of our public meetings. You can also check out this wiki to see more ways to participate in the evolution of WebExtensions.

Developer support for changes in add-on development

As you may have heard, there are a lot of changes coming up for add-on development. By the end of 2017, we will transition to WebExtensions as the standard for creating add-ons. Over the same period of time, existing methods for add-on development such as XUL/XPCOM will be deprecated. Multi-process Firefox (aka Electrolysis, or e10s) is also rolling out, which means some developers will have to update their add-on more than once. All this on the heels of add-on signing.

In the past few weeks, we’ve been gathering all the information and resources we could to help developers navigate the upcoming changes. We’re also lining up new content so we can continue to keep everyone informed and supported during this adjustment period.

Everything we’ve collected so far is here. A few notable pieces of information you’ll find:

  • A lookup tool you can use to see if your add-on will be impacted, and if so what the recommended migration path is.
  • A survey you can use to tell us which WebExtension APIs you need, so we can better prioritize them.
  • A timeline of changes that includes dates to the best of our knowledge. This is a working doc and will be updated as we learn of new information.
  • A calendar you can add to see upcoming meetings, blog posts, and office hours. We will also be adding release milestones to the calendar in the coming weeks.

Blog posts and other documentation related to the transition are added here as they’re created. We have more content queued up, and as you can see we could use even more. If you’re interested in writing about your experience creating a WebExtension add-on, or becoming e10s compatible, or anything else that others might find relevant, please sign up here.

We’ll keep releasing information on the wiki as they become available. If you have ideas or would like to pitch in, please reach out to us. We’re here to help.

AMO updates on version number re-use

With recent changes, addons.mozilla.org (AMO) now more strictly enforces our rule requiring version numbers for add-ons to be unique.

Previously, the website allowed some edge cases where if a version was deleted or disabled before we reviewed it, a new version with the same version number could be uploaded.  This caused issues with our CDN where the old xpi was cached. The hash (used for verification) could be wrong so the install would fail, and any users who installed the old xpi would fail to get the update to the ‘real’ version.

Now, once a version has been uploaded to AMO with a version number, you will be unable to use it again. Deleting or disabling the version won’t release the version number for re-use.  This restriction applies to add-ons that are uploaded for signing and external listing as well as ones listed on AMO.

We’re aware this will cause some irritation to developers when they try to upload again, and that perfect version “10.0.0.0” may be lost due to a hasty upload, but we believe the user benefits in this case outweigh the extra restriction on developers.

Add-ons Update – Week of 2016/03/09

I post these updates every 3 weeks to inform add-on developers about the status of the review queues, add-on compatibility, and other happenings in the add-ons world.

The Review Queues

In the past 3 weeks, 1065 add-ons were reviewed:

  • 1007 (95%) were reviewed in less than 5 days.
  • 26 (2%) were reviewed between 5 and 10 days.
  • 32 (3%) were reviewed after more than 10 days.

There are 102 listed add-ons awaiting review.

You can read about the recent improvements in the review queues here.

If you’re an add-on developer and are looking for contribution opportunities, please consider joining us. Add-on reviewers get invited to Mozilla events and earn cool gear with their work. Visit our wiki page for more information.

Firefox Accounts

Firefox Accounts is now live on AMO. Next time you log in, you’ll be prompted to migrate your account. If you have any problems with this process, please post in the forum thread.

Firefox 45 Compatibility

This compatibility blog post is up. The bulk compatibility validation was run last week.

Firefox 46 Compatibility

The compatibility blog post is up. We expect to run the bulk validation in a couple of weeks.

As always, we recommend that you test your add-ons on Beta and Firefox Developer Edition to make sure that they continue to work correctly. End users can install the Add-on Compatibility Reporter to identify and report any add-ons that aren’t working anymore.

Extension Signing

The wiki page on Extension Signing has information about the timeline, as well as responses to some frequently asked questions. The current plan is to remove the signing override preference in Firefox 46 (updated from the previous deadline of Firefox 44).

Electrolysis

Electrolysis, also known as e10s, is the next major compatibility change coming to Firefox. Firefox will run on multiple processes now, running content code in a different process than browser code.

This is the time to test your add-ons and make sure they continue working in Firefox. We’re holding regular office hours to help you work on your add-ons, so please drop in on Tuesdays and chat with us!

WebExtensions

We’re working on the new WebExtensions API, and we recommend that you start looking into it for your add-ons. You can track progress of its development in http://www.arewewebextensionsyet.com/.

Your Story Goes Here

011How do add-ons enhance your browsing experience in uniquely productive or creative ways?

We’re planning to run a series of stories on this blog on how people use add-ons to make Firefox their own. Maybe you’re one of these folks? If so, we want to speak with you!

The stories can be about anything… students who use calculator, language, or other education content… professionals who use add-ons to help run their businesses… artists leveraging the power of creativity tools… serious online shoppers using add-ons to scour the web for great deals… so many possibilities!

If you’d like to tell your story, please email editor@mozilla.com with “my story” in the subject line and tell us a bit about the add-ons you use and why.

Breakthroughs in the add-on review queues

The last few months have been dense with big changes in the add-ons world. Just reading this blog and looking at all the new faces and posts should give you a good idea. Today I’d like to show you how these changes have affected the add-on review queues and what you can expect for the future.

These charts show the state of the review queues on a weekly basis, starting on Q4 last year:

Full review queue Preliminary review queue Update queue

The green area represents add-ons that have been waiting under 5 days, yellow between 5 and 10 days, and red over 10 days.

As you can see, we were doing pretty poorly in late 2015. We only had one admin reviewer, Andreas Wagner, whose attention was almost entirely focused on reviewing unlisted add-ons (not included in the charts). We already had a significant backlog of add-ons awaiting admin review, and during this time it got a lot worse. We were looking for new admin reviewers we could get on board as contractors, but most of our attention was focused on extension signing.

Turning the tide

A few things happened that helped us get the review queues under control:

  • We stopped gating unlisted add-on signing by review. This freed Andreas’s time so he could go back to review listed add-ons.
  • Our team (finally!) grew. We now have Philipp Kewisch as our second full time admin reviewer and Noitidart helping part time also as an admin. Other areas of the team also got help, like the AMO dev team and editorial, all of which has afforded us much-needed breathing room and focus.
  • Our volunteer team also ramped up their efforts and kept up with the increased submission rate. During the first few weeks of the year, they averaged over 500 reviews per week! This is an amazing feat, especially to those of us who know about the pains of code review.

The review queues are now almost entirely in the green, which is fantastic (in fact, we’re all green in today’s report). However, it’s reasonable to have some skepticism, since we’ve been here before. We rely on a couple of very active volunteers who do the bulk of the reviews. And we will need to turn more of our attention to unlisted submissions soon (though not to the same extent as before).

Keeping it that way

So, what are we doing to maintain wait times that developers can rely on?

First, we have the staff additions I mentioned earlier. Having a stable admin team contributes greatly because we have a higher baseline even when volunteer activity drops. We also have another part-time opening that we will try to fill soon.

We’ve made some changes to the reviewer tools to encourage more participation from less-active reviewers. The review team has grown significantly in the past couple of months, and we receive new applications all the time. Are you interested in joining us? Please apply!

Finally, I think we’re past the big wave of submissions that happened due to the introduction of extension signing. But we do have a stable and steady increase in developer activity (which is good!).

We’d like to thank the add-on developer community for their immense patience during this time. We know it’s been rough and we’re very grateful that you stuck with us. We’ll keep working hard to make sure this becomes the standard.

March 2016 Featured Add-ons

Pick of the Month: Tab Memory Usage

by Jeremy Schomery
Tab Memory Usage displays the amount of memory used in each of your open tabs.

“Works for me just like the name says, displays memory usage next to the tab name. Simple, awesome.”

Featured: ImTranslator

by Smart Link Corporation
Choose translations in 70+ languages by comparing results from Google, Microsoft, and Babylon translation services. Features also include text-to-speech, dictionary, and back translations.

“I’m on the Web 12 hours a day using Firefox, and I have ImTranslator running all the time.”

Friend of Add-ons: Johann Hofmann

Our newest Friend of Add-ons is Johann Hofmann! Johann is active in the Rust ecosystem, and has been contributing to WebExtensions in the past few months. He explains, “I like that the WebExtensions project enables me to have such a big impact on the API that I and many other developers will use in the future.”

Johann has also contributed code to JPM, and is a volunteer AMO reviewer. He enjoys open source projects and explains how he began contributing to add-ons:

“I got into contributing to add-ons when I started to write my own Firefox extensions and noticed small bugs in the tooling, which I managed to fix. I love contributing to open source. I think Mozilla is the perfect place to do open source because it focuses on the people behind the code. Everyone is really welcoming and trying to help you make an impact, in real-life events as much as on IRC.”

Thanks to Johann for making an impact with add-ons.

We encourage you to document your contributions on the Recognition page, which also serves as the nomination vehicle for the Friends of Add-ons features.

How to get a quicker review for add-ons with source code attached

When submitting an add-on to addons.mozilla.org (AMO), it is sometimes necessary to attach source code in addition to the xpi file. This usually applies to add-ons with obfuscated code, because a reviewer wouldn’t be able to approve an add-on without reviewing what was obfuscated. Since these types of add-ons are more complex to review, I’ve written some tips on what you can do to help them get through the queue faster.

When to attach sources

When the add-on xpi file you upload to addons.mozilla.org (AMO) contains code that is not completely readable by a human, it is probably a good idea to attach sources.

For example, if you used tools like uglifyJS, Google Closure Compiler, browserify or a custom pre-processor, you will have to upload sources. The same goes if you are using js-ctypes or including other binary components.

There are also a few cases where it is actually NOT required and in fact not recommended. If your library only contains third-party minified libraries (like jQuery or Angular), or if the libraries you are calling via js-ctypes are system libraries or open source, please do not upload sources. Instead, provide links to the repositories of the respective libraries.

What happens during reviews

When you attach sources, the add-on is marked for “admin-review”. This means that your sources and are only accessible to a small group of admins. We do this to protect your sources.

A very important aspect of reviewing sources is reproducing the obfuscation. As we need to treat every extension developer the same, we must verify that the source code we reviewed matches the uploaded xpi. If we skip this step, a malware author could provide us with legitimate-looking sources and add a backdoor to the previously minified xpi file.

Here are the steps we take:

  1. Download the sources and extract them.
  2. Run <magic steps>, including minifiers, obfuscators, compilers, or code generators.
  3. Take the output directory from the previous step and compare it with the add-on xpi that has been uploaded.
  4. Review the source code files as we would review any other add-on that does not have sources.

In step 3, we use a diff tool to compare the generated sources to the add-on xpi file. There must be no differences at all. To save time, it is very important to provide us with all the <magic steps>. If you don’t add this, we will have to get in touch with you, and that adds time to the review process.

Providing instructions

The easiest way for you to provide the magic steps is to include a README file in the uploaded sources. If it is just one or two files that are obfuscated, the instructions can be something like “run uglifyjs data/mycoolstuff.js”. If the extension is any more complex, please provide a script that we can run that takes care of everything at once. Things you should mention in your README include:

  • Prerequisites that need to be downloaded separately, for example yuicompressor.
  • For less popular or custom-written build tools, provide links where they can be downloaded, as well as installation steps.
  • If a specific version of supplemental software needs to be used, please let us know. But avoid doing so if the latest version would work just as well.
  • All commands we should run to go from sources to a generated xpi file that matches the one you’ve uploaded, for example npm install or a grunt target.

Please assume the reviewer has a vanilla operating system set up. You don’t need to describe how to install common tools including npm, node, the add-on SDK, but please make sure the reviewer can figure out how to install everything needed to generate the xpi file.

Aside from the README file, you also need to package everything required to build. If your add-on depends on a private repository or frameworks not commonly available, please include them as well.

Desired outcome

Common build tools used are make, grunt, gulp and ant. If you don’t already have a build target that runs the above steps, please add a target that does so. For example, allow us to run grunt firefox-dist to create the generated xpi. Here is an example of what the sources could look like. The dist directory is initially empty, until your build script (in this case grunt) generates the directory contents. You could then zip them up and upload them to AMO.

sources
├── README.md                 
├── Gruntfile.js              
├── package.json               
├──────────────────────────────── dist
│                                 ├── bootstrap.js
│                                 ├── install.rdf
│                                 ├── package.json
├── data                          ├── data
│   ├── js                        │   ├── js
│   │   ├── dialog.js             │   │   ├── dialog.min.js
│   │   └── popup.js              │   │   └── popup.min.js
│   ├── scss                      │   ├── css
│   │   ├── popup.scss            │   │   ├── popup.css
│   │   ├── dialog.scss           │   │   └── dialog.scss
│   │   └── common.scss           │   │
│   ├── html                      │   ├── html
│   │   ├── dialog.html           │   │   ├── dialog.html
│   │   └── popup.html            │   │   └── popup.html
│   ├── vendor                    │   ├── vendor
│   │   └── jquery-2.1.4.min.js   │   │   └── jquery-2.1.4.min.js
│   └── images                    │   └── images
│       └── logo.png              │       └── logo.png
├── lib                           ├── lib
│   └── main.js                   │   └── main.js
└── locale                        └── locale
    └── en-US.properties              └── en-US.properties

 

A few more tips

If you can avoid obfuscating or minifying code, your review can be done by any reviewer. Should you still need to attach sources, make sure you provide clear instructions so that our admin reviewers can handle your add-on quickly. Responding to questions quickly and using well-known obfuscation tools also improve review time.

Some last words

While our reviewers, both volunteer and staff, review add-ons around the clock, there may be times when it just takes longer. This can be due to anything from Firefox releases and major product changes, to holiday seasons.

I hope you’ve found this post helpful. There’s a lot to remember, but after you’ve done this once or twice you should get the hang of it. If you’d like a page to bookmark that contains this information and some more details on the topic, please head over to our new article on MDN.

If you also have tips to share, or questions on this topic, please post in our forums. Also, if you ever want to sit with developers on the other side of the table, perhaps consider applying to become an add-on reviewer?

Add-on Compatibility for Firefox 46

Firefox 46 will be released on April 19th 26th. Here’s the list of changes that went into this version that can affect add-on compatibility. There is more information available in Firefox 46 for Developers, so you should also give it a look.

General

Signing

  • Firefox is currently enforcing add-on signing, with a preference to override it. Firefox 46 will remove the preference entirely , which means your add-on will need to be signed in order to run in release versions of Firefox. You can read about your options here.

New

Let me know in the comments if there’s anything missing or incorrect on these lists. If your add-on breaks on Firefox 46, I’d like to know.

The automatic compatibility validation and upgrade for add-ons on AMO will happen in a few weeks (the one for Firefox 45 is still pending), so keep an eye on your email if you have an add-on listed on our site with its compatibility set to Firefox 45.