Developing Extensions With Web-ext 1.0

As the transition to WebExtensions continues, we are also building tools to make developing them faster and easier than ever. Our latest is a command line tool called web-ext, which we released recently as an initial working version with some basic features.

We built it because we want developing extensions to be fast and easy. Just as Firefox’s WebExtensions API is designed for cross-browser compatibility, we want web-ext to eventually support platforms such as Chrome or Opera. We will continue developing jpm in parallel, as needed.

To give it a try, you can install it from npm:

npm install --global web-ext

When developing an extension, here’s how you can run it in Firefox to test it out:

cd /path/to/your/source
web-ext run

This is similar to how you can load your source directly on the about:debugging page.

When you’ve got your extension working, here’s how to build an XPI file that can be submitted to addons.mozilla.org:

web-ext build

You can also self-host your XPI file for distribution but it needs to be signed by Mozilla first. Here’s how to build and sign an XPI file:

web-ext sign

The end user documentation is a work in progress but you can reference all commands and options by typing:

web-ext --help

As you can see, this is a very early release just to get you started while we continue adding features. If you’d like to help out on the development of web-ext, check out the contributor guide and take a look at some good first bugs.

How an Add-on Saved a Student Untold Hours

This is the actual hand Rami uses to click his mouse and save text to files.

This is the actual hand Rami uses to click his mouse and save text to files.

Rami Khader is pursuing a Master’s degree in Information Technology Management at Regis University. This endeavor involves exhaustive research and writing. Early in his studies, Rami encountered a clear need when it came to organizing and recalling online literature reviews for his various papers and thesis.

At the start of any research project, Rami says he reviews at least 30 other papers related to his subject. But only small, scattered portions of any paper bear relation to his own topic. “My issue was—I needed a quick tool to copy a summary of text I found, and then add a data time stamp and URL reference so I could return back to the website,” explains Rami. “I was doing this manually and it consumed lots of time.”

Then he found Save Text to File, an add-on that saves selected text and its metadata to a directory file with one simple right-click of the mouse.

“This was exactly what I was looking for,” says Rami. “For each chapter of my thesis I have a file full of summarized text and URLs of the papers that I can quickly access and remember what each page and paper was about. I have my own annotated bibliography that’s ready to be formatted and added to my papers.”

Very cool. Thanks for sharing your add-ons story with us, Rami!

If you, kind reader, use add-ons in interesting ways and want to share your experience, please email us at editor@mozilla.com with “my story” in the subject line.

The “Why” of Electrolysis

A multi-process architecture is finally coming to Firefox. Known by its codename, “Electrolysis” or “e10s,” this project aims to split the Firefox browser into a single process for the UI, and several processes for web content, media playback, plugins, etc.

Electrolysis represents a long overdue modernization of Firefox’s codebase: Internet Explorer has been multi-process since IE 8 Beta 1 in early 2008, Chrome followed six months later, and even Safari went multi-process in the summer of 2011. Put another way, Firefox has been the only major browser using a single process architecture for the past five years.

Benefits for Users

Though the result should be visually indistinguishable from a single-process Firefox, the multi-process model offers many compelling advantages in responsiveness, stability, performance, and security:

  • Responsiveness. By placing web content into a separate process from the browser interface, we can ensure that Firefox’s UI stays snappy even when a page is doing heavy computation or undergoing garbage collection.
  • Stability. Tabs shouldn’t crash, but when they do, isolating web content in separate processes means they can crash without taking down the rest of the browser.
  • Performance. A multi-process architecture lets us spread work across the many CPU cores present in modern computers and smartphones, resulting in more efficient utilization of the underlying hardware.
  • Security. Process separation makes it easier to implement restrictive security sandboxes for web content.

A common concern is that switching to a multi-processes architecture will dramatically increase Firefox’s memory usage. This is not the case. While multiple processes will have a greater memory footprint than a single process, the impact should be limited: we’re currently seeing multi-process Firefox use 10-20% more memory, however, it still uses half the memory of Chrome with the same sites loaded.

To ensure we don’t consume too much RAM, the first release of e10s will only use a single additional process for web content. We’ll add more processes in subsequent releases, as we become more memory efficient.

Challenges for Add-on Developers

The move to multi-process is an investment in the future: we’re paying down technical debt and redesigning Firefox’s architecture at a fundamental level. Like any change of this magnitude, there are associated challenges:

1. Cross-Process Communication. Before e10s, add-ons were accustomed to having direct access to both the browser context and web page content. With e10s, those two contexts are now two separate processes which must communicate through asynchronous message passing.

Many, but not all, add-ons will require modification to work efficiently with this new design. Add-ons that do not access web content, only use high-level SDK APIs, or are written with the new WebExtension APIs will not need modification.

You can see the compatibility status of popular add-ons at AreWeE10SYet.com, or use the compatibility checker to see what changes your add-on might need.

2. Performance. To help with compatibility during the transition to multi-process, we’ve implemented several temporary shims that allow backwards-compatible, synchronous communication between the browser and web content. This lets add-ons work as if Firefox was still single-process, but with potentially worse performance than before, since these shims must completely block multiple processes and incur overhead for inter-process communication.

The shims are available to add-ons by default, but authors should always prefer asynchronous message passing, high-level SDK APIs, or WebExtensions instead of shims.

You can disable shims for your add-on by setting the multiprocess permission to true in your package.json, or set <em:multiprocessCompatible> to true in your install.rdf. This is how we know your add-on is e10s-compatible and will ensure that it won’t be marked as incompatible when e10s ships with Firefox.

Try it Today

Multi-process has been enabled by default in Firefox Developer Edition since version 42, released last August. If you’re an add-on developer or Firefox enthusiast, we highly recommend you try it out and report any bugs you encounter, either in Bugzilla or in the comments below. The Mozilla Developer Network also hosts extensive documentation on multi-process Firefox, including migration guides for add-on authors. Check it out, and let us know what you think!

Add-on Compatibility for Firefox 47

Firefox 47 will be released on June 7th. Here’s the list of changes that went into this version that can affect add-on compatibility. There is more information available in Firefox 47 for Developers, so you should also give it a look.

General

Tabs

XPCOM

New

Let me know in the comments if there’s anything missing or incorrect on these lists. If your add-on breaks on Firefox 47, I’d like to know.

The automatic compatibility validation and upgrade for add-ons on AMO will happen in a few weeks, so keep an eye on your email if you have an add-on listed on our site with its compatibility set to Firefox 46.

Improving Review Time by Providing Links to Third Party Sources

Earlier I wrote an article about submitting add-ons with sources attached, which is primarily needed for obfuscated or minified add-ons. I only briefly mentioned the specific cases where attaching sources is in fact NOT recommended. If you’ve determined you do not need to upload sources but you still want to profit from blazing fast review speeds, please read on.

To reiterate, you do not need to upload source code if your add-on only contains (minified) third-party libraries, or if the libraries you are calling via js-ctypes are system libraries or open source. What you need to do instead is provide links to the exact file versions in the repositories of the respective libraries. You should add these links in the “Notes to Reviewers” section when viewing or uploading your new version.

Just as with your add-on code, we review third-party library code for potential issues. When we notice a library is used more often, we can add the (sha256) checksum to a list of known libraries that are specially marked in the editor tools. Reviewers do not need to review this library again, which saves a lot of time.

The downside to using checksums is that the file must match exactly, down to the byte. A common issue we encounter is libraries being retrieved from non-official sites, for example CDNs. These sites may make slight changes, often something simple like white spaces, or adding/removing a source map URL. Similarly, some developers copy/paste the libraries into a text editor, which may result in different line endings.

Now to the important part: how to specify the repository links. You don’t need to do this for the common libraries mentioned in our list (currently: angular, backbone, backbone.localStorage, crypto.js, dojo, jquery, jqueryui, moment, mootools, prototype.js, scriptaculous, swfobject.js, underscore, yui). For all other libraries, we need the link to an exact copy of the file you have submitted, from the official website or repository. Here is an example:

Let’s say you are using the minified version of mousetrap version 1.4.2 because you haven’t had the chance to update to the latest version.

If the library is on github, you can usually find this version under the “releases” link, then clicking on the small tag icon next to the version number, then navigating to the file in the repository. For bonus points, if you are using the minifed version, also provide a link to the unminifed version beside it. If the library does not use github releases but instead just gives you a zip to download, provide the link to the zip instead.

Mentioning the links upfront will shorten the review time since we can check the library right away instead of asking you about it, making reviews faster. Leave a comment if you have any questions!

April 2016 Featured Add-ons

Pick of the Month: WhatsApp™ Desktop

by Elen Norphen
Enjoy easy access to WhatsApp right from your browser, including incoming message notifications.

”Ahhhh sweet extension!”

Featured: Video Downloader Prime

by Mark
Simple download process for most popular video formats.

”Very easy to work with. It captures video very fast!”

Featured: Google Translate Anywhere

by Jeremy Schomery
Provides a floating multilingual panel for any word or phrase you highlight.

”Everything I want in a dictionary extension: fast pop-up translations, a button for listening to pronunciations, and also definitions. Great job!”

Nominate your favorite add-ons

Featured add-ons are selected by a community board made up of add-on developers, users, and fans. Board members change every six months, so there’s always an opportunity to participate. Stayed tuned to this blog for the next call for applications. Here’s further information on AMO’s featured content policies.

If you’d like to nominate an add-on for featuring, please send it to amo-featured@mozilla.org for the board’s consideration. We welcome you to submit your own add-on!

Tinkering with WebExtensions

I’ve been writing about WebExtensions development on my blog. I’ve kept those posts over there because they’re short one-offs that I think would be too noisy for this blog and its wide audience.

That’s why I’m giving you this quick overview of what I’ve been writing so you can give them a closer look if you’re interested. The first two posts might interest you if you’re an add-on developer curious about WebExtensions. The second two are more meta, and touch on documentation, code review, and how we’re trying to shape the developer experience.

Let us know if there’s any topic around WebExtensions or other add-on development you want us to cover.

Add-ons Update – Week of 2016/03/30

I post these updates every 3 weeks to inform add-on developers about the status of the review queues, add-on compatibility, and other happenings in the add-ons world.

The Review Queues

In the past 3 weeks, 1046 add-ons were reviewed:

  • 981 (94%) were reviewed in fewer than 5 days.
  • 37 (4%) were reviewed between 5 and 10 days.
  • 28 (3%) were reviewed after more than 10 days.

There are 73 listed add-ons awaiting review.

You can read about the recent improvements in the review queues here.

If you’re an add-on developer and are looking for contribution opportunities, please consider joining us. Add-on reviewers get invited to Mozilla events and earn cool gear with their work. Visit our wiki page for more information.

Compatibility Communications

Most of you should have received an email from us about the future compatibility of your add-ons. You can use the compatibility tool to enter your add-on ID and get some info on what we think is the best path forward for your add-on.

To ensure long-term compatibility, we suggest you start looking into WebExtensions, or use the Add-ons SDK and try to stick to the high-level APIs. There are many XUL add-ons that require APIs that aren’t available in either of these options, which is why we’re also asking you to fill out this survey, so we know which APIs we should look into adding to WebExtensions.

Firefox 46 Compatibility

The compatibility blog post is up. The bulk validation was run yesterday.

Firefox 47 Compatibility

The compatibility blog post for 47 is coming soon.

As always, we recommend that you test your add-ons on Beta and Firefox Developer Edition to make sure that they continue to work correctly. End users can install the Add-on Compatibility Reporter to identify and report any add-ons that aren’t working anymore.

Extension Signing

The wiki page on Extension Signing has information about the timeline, as well as responses to some frequently asked questions. The current plan is to remove the signing override preference in Firefox 46.

Electrolysis

Electrolysis, also known as e10s, is the next major compatibility change coming to Firefox. Firefox will run on multiple processes now, running content code in a different process than browser code.

This is the time to test your add-ons and make sure they continue working in Firefox. We’re holding regular office hours to help you work on your add-ons, so please drop in on Tuesdays and chat with us!

Add-on Signing Enforcement In Firefox 46 for Android

As part of the Extension Signing initiative, signing enforcement will be enabled by default on Firefox 46 for Android. All Firefox for Android extensions on addons.mozilla.org have been signed, and developers who self-host add-ons for Android should ensure those add-ons are signed prior to April 12th, 2016.

Extension signing enforcement can be disabled using the preference outlined in the FAQ, and this preference will be removed in a future release.

More information on Extension Signing is available on the Mozilla Wiki, and information on the change to Firefox for Android can be found on Bugzilla under Bug 1244329.

Advantages of WebExtensions for Developers

First, a Little Backstory

Presently, Firefox supports two main kinds of add-ons. First were XUL or XPCOM add-ons, which interface directly with the browser’s internals. They are fabulously powerful, as powerful as the browser itself. However, with that power comes security risk and the likelihood that extensions will break as the browser changes.

The Add-on SDK was introduced to provide a stable abstraction API above browser internals to reduce extension compatibility issues between versions of Firefox and make it easier to review extensions for security. As the browser evolved, however, it became clear that we needed something even more robust to take us into the future.

WebExtensions is the new API for building add-ons in Firefox. It seeks to unify the extension APIs and architecture with those of other browsers in the name of interoperability and modern architecture.

The introduction of WebExtensions isn’t an arbitrary change—it stands to improve the stability and security of Firefox for users. It also has a number of advantages for developers.

Cross-Browser Interoperability

Potentially the most impactful aspect of WebExtensions is that it adopts the extension architecture used by browsers built on top of Chromium, notably Chrome and Opera. Mozilla is committed to implementing a large number of the individual APIs presently available to Chrome extensions. This means that it’s possible to have one codebase for an extension that will work in Firefox, Chrome, and Opera with a minimal amount of browser-specific code.

More than Chrome Parity

While our initial API priorities are focused on allowing Chrome extensions to interoperate with Firefox, we plan to competitively and actively expand the API capabilities of WebExtensions. While we anticipate the vast majority of Firefox Add-ons can be ported to WebExtensions, if you need access to browser components not presently exposed, the Add-on SDK is still available. If your add-on or one you use and love is doing something that is not possible with Chrome extensions or the Add-on SDK, please fill out this survey to help us prioritize new APIs.

Common Functionality is Easier

A large number of browser extensions take the form of content scripts, where Javascript is executed against certain (or in some cases all) web pages. WebExtensions optimizes for this case by allowing developers to reduce these types of extensions to 2 files: the script, and a manifest describing the extension and on which pages it should be injected.

Future-Proofing your Work

XPCOM/XUL add-ons are notoriously brittle, breaking whenever browser internals change. The Add-on SDK provides a more stable API surface, but has many synchronous APIs that cross between add-on contexts and web page contents. Mozilla is committed to moving Firefox to a multi-process architecture this year to increase security and stability (read more here), which will break many of these synchronous APIs. WebExtensions APIs are being implemented from the ground-up to be ready for a multi-process Firefox, using asynchronous message passing and event-driven interfaces to enable extensions to communicate with the browser and web pages.

More Secure Extensions

Because extensions built with the Add-on SDK can request XPCOM privileges, they could still introduce unintentional security and stability issues into Firefox. Even add-ons written by well-meaning developers can accidentally introduce vulnerabilities that could allow malicious code to execute with the full privileges of the browser. WebExtensions uses its manifest.json to mitigate this by requiring add-on authors to declare up front which permissions their code will need to operate. Unlike the Add-on SDK, WebExtensions does not allow arbitrary XUL/XPCOM access, so even insecure/vulnerable code is limited to its whitelisted subset of functionality. This vastly reduces the vulnerability surface of a WebExtension, leading to faster review times and a more stable browser.

The Path Forward

We know WebExtensions is a big change, and we’re committed to helping developers prepare and adapt as we roll it out. Initial support for WebExtensions is coming in Firefox 48, and add-ons built with WebExtensions can already be submitted to addons.mozilla.org. The time is now to start experimenting and porting your add-ons! Here are some links to help you get started:

  • If you’re an existing add-on author, check out this collection of resources on migrating your add-on to WebExtensions.
  • If you’re the author of a Chrome extension, we have a short guide on adding Firefox support to your existing codebase.
  • You can also check out this wiki to see more ways to participate in the evolution of WebExtensions.