WebExtensions for Firefox 49

Firefox 49 landed in Developer Edition this week, so we have another update on WebExtensions for you!

Since the release of Firefox 48, we feel WebExtensions are in a stable state. We recommend developers start to use the WebExtensions API for their add-on development. Since the last release, more than 35 bugs were closed on WebExtensions alone.

If you have authored an add-on in the past and are curious about how it’s affected by the upcoming changes, please use this lookup tool. There is also a wiki page and MDN articles filled with resources to support you through the changes.

APIs Implemented

The history API allows you to interact with the browser history. In Firefox 49 the APIs to add, query, and delete browser history have been merged. This is ideal for developers who want to build add-ons to manage user privacy.

In Firefox 48, Android support was merged, and in Firefox 49 support for some of the native elements has started to land. Firefox 49 on Android supports some of the pageAction APIs. This work lays the foundation for new APIs such as tabs, windows, browserAction in Android.

The WebNavigation API now supports the manual_subframe transitionType and keeps track of user interaction with the url bar appropriately. The downloads API now lets you download a blob created in a background script.

For a full list of bugs, please check out Bugzilla.

In progress

Things are a little bit quieter recently because there are things in progress that have absorbed a lot of developer time. They won’t land in the tree for Firefox 49, but we’ll keep you updated on their progress in later releases.

storage.sync

This API allows you to store some data for an add-on in Firefox and have it synced to another Firefox browser. It is most commonly used for storing add-on preferences, because it is not designed to be a robust data storage and syncing tool. For sync, we will use Firefox Accounts to authenticate users and enforce quota limits.

Whilst the API contains the word “sync” and it uses Firefox Accounts, it should be noted that it is different from Firefox Sync. In Firefox Sync there is an attempt to merge data and resolve conflicts. There is no similar logic in this API.

You can track the progress of storage.sync in Bugzilla.

runtime.connectNative

This API allows you to communicate with other processes on the host’s operating system. It’s a commonly used API for password managers and security software which needs to communicate with external processes.

To communicate with a native process, there’s a two-step process. First, your installer needs to install a JSON manifest file at an appropriate file location on the target computer. That JSON manifest provides the link between Firefox and the process. Secondly, the user installs the add-on. Then the add-on can call the connectNative, sendNativeMessage and other APIs:

chrome.runtime.sendNativeMessage('your-application',
  { text: "Hello" },
  function(response) {
    console.log("Received " + response);
});

Firefox will start the process if it hasn’t started already, and pipe commands through to the process. Follow along with the progress of runtime.connectNative on Bugzilla.

WebExtensions transition

With these ongoing improvements, we realise there are lots of add-ons that might want to start moving towards WebExtensions and utilise the new APIs.

To allow this, you will soon be able to embed a WebExtension inside an add-on. This allows you to message the WebExtension add-on.

The following example works with SDK add-ons, but this should work with any bootstrapped add-on. Inside your SDK add-on you’d have a directory called webextensions containing a full-fledged WebExtension. In the background page of that WebExtension will be the following:

chrome.runtime.sendMessage("test message", (reply) => {
  console.log("embedded webext got a reply", reply);
});

Then you’d be able to reply in the SDK add-on:

var { api } = require('sdk/webextension');
api.onMessage.addListener((msg, sender, sendReply) =>
  console.log("SDK add-on got a message", {msg,sender}); 
  sendReply("reply");
});

This demonstrates sending a message from the WebExtension to the SDK add-on.Persistent bi-directional ports will also be available.

Using this technique, we hope add-on developers can leverage WebExtensions APIs as they start migrating their add-on over to WebExtensions. Follow along with the progress of communication on Bugzilla.

There are also lots of other ways to get involved with WebExtensions, so please check them out!

Developer profile: Luís Miguel (aka Quicksaver)

Luís Miguel claims to "bleed" add-ons.

“I bleed add-on code,” says Luís Miguel.

Always a heavy user of bookmarks and feeds, Luís Miguel one day decided to customize his Firefox sidebar to better accommodate his personal tastes (he didn’t like how the ever-present sidebar pushed web content aside). But Luís soon discovered he could only go so far solving this problem with minor CSS tweaks.

That’s when he resolved himself to develop a full-blown add-on. He created OmniSidebar, an elegant extension that allows you to slide your sidebar into view with a simple gesture, among other rich interface features.

Another of Luís’ add-ons—FindBar Tweak—has a similar origin story; Luís identified a personal desire first, then built an add-on to address it. In this case he wanted a quick way to scan code for just the parts applicable to his work.

“Earlier today I was working with pinned tabs, and I had to research how they happen,” explains Luís. “Looking for the word ‘pinned’ in Firefox’s code using FindBar Tweak, I was able to easily read through only the relevant bits in a few seconds, because it not only takes me directly to what I’m looking for, but also helps me visually make sense of what is actually relevant.”

Luís has created four other add-ons, including the popular Tab Groups. Luís spoke with me briefly about his unique interest in add-ons…

It sounds like you basically create all of your add-ons based on personal browsing needs. Is that an accurate characterization?
Luís Miguel: Yeah, that’s a fair way to put it. Everything has been about making my Firefox behave and interact with me the way I prefer it to, from quickly finding what I’m looking for in a page, to having my feeds and bookmarks readily accessible without screwing up the rest of the page layout, properly organizing all my buttons in the browser, keeping a clean window but always ready to interact in whatever way I need it to. The “anything goes” approach of Firefox add-ons make them a very fun playground for me in that respect.

As a procrastination enthusiast, I’ve found that when the process—towards whatever goal—is optimal, fast, and effective, I am the most motivated to do it. So I look for every way to enhance productivity in every task I do, not just for developing, even just casual browsing really.

Puzzle Bars can conveniently pull up your customized toolbar with a simple swing of the mouse.

Puzzle Bars conveniently pulls up your customized toolbar with a simple swing of the mouse.

Although, that’s not entirely the case anymore. There’s a lot in my add-ons that come from user suggestions and requests, many [features] I don’t really use myself. But if it’s to improve the add-on and I have the availability to do it, how can I say no to the most awesome and supportive users a developer could ask for?

Beyond add-ons you’ve created, do you do other types of development?
LM: I’ve contributed a few patches for Firefox itself through Bugzilla, but that’s about it. When I set out to do something, I focus on it, so right now I bleed add-on code.

Are you working on any new add-ons you want to tell us about? 
LM: I do have a couple of ideas for new add-ons, but those probably won’t happen for a while, mostly because I have to focus on my current ones a lot, as they will need to be ported to WebExtensions eventually. Since my add-ons are mostly UI-modifications at their base, that will be… challenging.

I have been working on Tab Groups 2 though, which is a major rewrite of almost everything, to hopefully make it more stable and perform better. It’s turning out really awesome.

What other add-ons—ones you haven’t created—are your favorites and why?
LM: I have around 30 add-ons enabled in my main profile at any one time. A few favorites come to mind…

AdBlock Plus is always at the top of my list. I dislike waiting a long time for pages to fully load because of several blinking boxes, or misleading ads.

Download Panel Tweaker for a sleek, direct, and fully functional downloads panel.

Tab Mix Plus for optimized opening, closing, moving, and loading tab behavior. There’s just so much it can do!

Stylish for a few nits here and there, and the custom glass style I made for myself, which I’ve found no other theme or add-on out there can even come close to this.

I especially like Turn Off the Lights. It helps me focus on whatever video I’m watching. I tend to get distracted easily. If there’s something in the corner of my eye that stands out for some reason, I will look at it. So that functionality is simple and yet very effective.

Thanks for chatting with us, Luís! I can think of no better way to close this conversation than by sharing a recent installment of your web series…

Add-ons Update – Week of 2016/06/01

I post these updates every 3 weeks to inform add-on developers about the status of the review queues, add-on compatibility, and other happenings in the add-ons world.

The Review Queues

In the past 3 weeks, 1277 listed add-ons were reviewed:

  • 1192 (93%) were reviewed in fewer than 5 days.
  • 68 (5%) were reviewed between 5 and 10 days.
  • 17 (1%) were reviewed after more than 10 days.

There are 67 listed add-ons awaiting review.

You can read about the recent improvements in the review queues here.

If you’re an add-on developer and are looking for contribution opportunities, please consider joining us. Add-on reviewers get invited to Mozilla events and earn cool gear with their work. Visit our wiki page for more information.

Firefox 48 Compatibility

The compatibility blog post for Firefox 48 is up. The bulk validation should be run in the coming weeks.

As always, we recommend that you test your add-ons on Beta and Firefox Developer Edition to make sure that they continue to work correctly. End users can install the Add-on Compatibility Reporter to identify and report any add-ons that aren’t working anymore.

Extension Signing

The wiki page on Extension Signing has information about the timeline, as well as responses to some frequently asked questions. The current plan is to remove the signing override preference in Firefox 48. This was pushed back because we don’t have the unbranded builds needed for add-on developers to test on Beta and Release.

Multiprocess Firefox

If your add-on isn’t compatible with multiprocess Firefox (e10s) yet, please read this guide. Note that in the future Firefox will only support extensions that explicitly state multiprocess compatibility, so make sure you set the right flag:

June 2016 Featured Add-ons

Firefox Logo on blue background

Pick of the Month: Smart HTTPS (Encrypt Your Communications)

by ilGur
Automatically changes HTTP addresses to secure HTTPS, and if loading errors occur, will revert back to HTTP.

“Using it is as easy as installing.”

Featured: YouTube No Buffer (Stop Autoplaying)

by James Fray
A lightweight extension that prevents YouTube from auto-buffering videos.

“I needed this, as YouTube makes the browser too slow. Thanks.”

Featured: Weather Forecast Plus

by Alexis Jaksone
Easily check the weather anywhere in the world right from the convenience of your toolbar.

“Simple, unobtrusive, snappy.”

Nominate your favorite add-ons

Featured add-ons are selected by a community board made up of add-on developers, users, and fans. Board members change every six months, so there’s always an opportunity to participate. Stayed tuned to this blog for the next call for applications. Here’s further information on AMO’s featured content policies.

If you’d like to nominate an add-on for featuring, please send it to amo-featured@mozilla.org for the board’s consideration. We welcome you to submit your own add-on!

Using Google Analytics in Extensions

As an add-on developer, you may want to have usage reporting integrated into your add-on. This allows you to understand how your users are using the add-on in real life, which can often lead to important insights and code updates that improve user experience.

The most popular way to do this is to inject the Google Analytics script into your codebase as if it were a web page. However, this is incompatible with our review policies. Injecting remote scripts into privileged code – or even content – is very dangerous and it’s not allowed. Fortunately, there are ways to send reports to Google Analytics without resorting to remote script injection, and they are very easy to implement.

Update: add-ons that use GA are required to have a privacy policy on AMO, and the data they send should be only what’s strictly necessary for usage reporting. This blog post is meant to show the safer ways of using GA, not advocate its unrestricted use.

I created a branch of one of my add-ons to serve as a demo. The add-on is a WebExtension that injects a content script into some AMO pages to add links that are useful to admins and reviewers. The diff for the branch shouldn’t take much time to read and understand. It mostly comes down to this XHR:

let request = new XMLHttpRequest();
let message =
  "v=1&tid=" + GA_TRACKING_ID + "&cid= " + GA_CLIENT_ID + "&aip=1" +
  "&ds=add-on&t=event&ec=AAA&ea=" + aType;

request.open("POST", "https://www.google-analytics.com/collect", true);
request.send(message);

In my demo I do everything from the content script code. For add-ons with a more complex structure, you should set up a background script that does the reporting and have content scripts send messages to it if needed.

I set up my reporting so the hits are sent as events. You can read about reporting types and all the different parameters in the Google Analytics developer docs.

Thanks to the real-time tracking feature I could see that my implementation was working right away:

live-tracking

That’s it! With just a few lines of code you can set up Google Analytics for your add-on safely, in a way that meets our review guidelines.

Add-on Compatibility for Firefox 48

Firefox 48 will be released on August 2nd. Here’s the list of changes that went into this version that can affect add-on compatibility. There is more information available in Firefox 48 for Developers, so you should also give it a look.

General

XPCOM and Modules

New

Let me know in the comments if there’s anything missing or incorrect on these lists. If your add-on breaks on Firefox 48, I’d like to know.

The automatic compatibility validation and upgrade for add-ons on AMO will happen in a few weeks, so keep an eye on your email if you have an add-on listed on our site with its compatibility set to Firefox 47.

AMO technical architecture

addons.mozilla.org (AMO) has been around for more than 12 years, making it one of the oldest websites at Mozilla. It celebrated its 10th anniversary a couple of years ago, as Wil blogged about.

AMO started as a PHP site that grew and grew as new pieces of functionality were bolted on. In October 2009 the rewrite from PHP to Python began. New features were added, the site grew ever larger, and now a few cracks are starting to appear. These are merely the result of a site that has lots of features and functionality and has been around for a long time.

The site architecture is currently something like below, but please note this simplifies the site and ignores the complexities of AWS, the CDN and other parts of the site.

Basically, all the code is one repository and the main application (a Django app) is responsible for generating everything—from HTML, to emails, to APIs, and it all gets deployed at the same time. There’s a few problems with this:

  • The amount of functionality in the site has caused such a growth in interactions between the features that it is harder and harder to test.
  • Large JavaScript parts of the site have no automated testing.
  • The JavaScript and CSS spill over between different parts of the site, so changes in one regularly break other parts of the site.
  • Not all parts of the site have the same expectation of uptime but are all deployed at the same time.
  • Not all parts of the site have the same requirements for code contributions.

We are moving towards a new model similar to the one used for Firefox Marketplace. Whereas Marketplace built its own front-end framework, we are going to be using React on the front end.

The end result will start look something like this:

image00

A separate version of the site is rendered for the different use cases, for example developers or users. In this case a request comes in hits one of the appropriate front-end stacks. That will render the site using React universal in node.js on the server. It will access the data store by calling the appropriate Python REST APIs.

In this scenario, the legacy Python code will migrate to being a REST API that manages storage, transactions, workflow, permissions and the like. All the front-facing user interface work will be done in React and be independent from each other as much as possible.

It’s not quite micro services, but the breaking of a larger site into smaller independent pieces. The first part of this is happening with the “discovery pane” (accessible at about:addons). This is our first project using this infrastructure, which features a new streamlined way to install add-ons with a new technical architecture to serve it to users.

As we roll out this new architecture we’ll be doing more blog posts, so if you’d like to get involved then join our mailing list or check out our repositories on Github.

Add-ons Update – Week of 2016/05/11

I post these updates every 3 weeks to inform add-on developers about the status of the review queues, add-on compatibility, and other happenings in the add-ons world.

The Review Queues

In the past 3 weeks, 1387 listed add-ons were reviewed:

  • 1314 (95%) were reviewed in fewer than 5 days.
  • 40 (3%) were reviewed between 5 and 10 days.
  • 33 (2%) were reviewed after more than 10 days.

There are 67 listed add-ons awaiting review.

You can read about the recent improvements in the review queues here.

If you’re an add-on developer and are looking for contribution opportunities, please consider joining us. Add-on reviewers get invited to Mozilla events and earn cool gear with their work. Visit our wiki page for more information.

Compatibility Communications

Most of you should have received an email from us about the future compatibility of your add-ons. You can use the compatibility tool to enter your add-on ID and get some info on what we think is the best path forward for your add-on. This tool only works for listed add-ons.

To ensure long-term compatibility, we suggest you start looking into WebExtensions, or use the Add-ons SDK and try to stick to the high-level APIs. There are many XUL add-ons that require APIs that aren’t available in either of these options, which is why we ran a survey so we know which APIs we should look into adding to WebExtensions. You can read about the survey results here.

We’re holding regular office hours for Multiprocess Firefox compatibility, to help you work on your add-ons, so please drop in on Tuesdays and chat with us!

Firefox 47 Compatibility

The compatibility blog post for 47 is up and the bulk validation has been run.

Firefox 48 Compatibility

The compatibility blog post for Firefox 48 is coming up soon.

As always, we recommend that you test your add-ons on Beta and Firefox Developer Edition to make sure that they continue to work correctly. End users can install the Add-on Compatibility Reporter to identify and report any add-ons that aren’t working anymore.

Extension Signing

The wiki page on Extension Signing has information about the timeline, as well as responses to some frequently asked questions. The current plan is to remove the signing override preference in Firefox 47.

The preference was actually removed recently in the beta channel (future Firefox 47), though this was done before the unbranded builds were available for testing. We’re trying to get those builds out as soon as possible to avoid more disruption. For now I suggest you use Developer Edition for testing or, if your add-on is restartless, you can also use the temporary load option.

Results of the WebExtensions API Survey

In March, we released a survey asking add-on developers which APIs they need to transition successfully to WebExtensions. So far, 235 people have responded, and we’ve summarized some of the findings in these slides.

Developers with the most add-ons responded at a disproportionate rate. Those with 7 or more add-ons represent 2% of the add-on developer community, and those with 4-6 add-ons represent 3%, but they comprised 36.2% of survey respondents. This didn’t come as a surprise, since the most active developers are also the most engaged and have the most to gain by migrating to WebExtensions.

How many add-ons have you worked on?

Nearly half of respondents have tried implementing their add-ons in Chrome, and the most cited limitation is that it’s restrictive. Developers could not do much with the UI other than add buttons or content tabs. We intend to offer APIs that give developers more freedom to customize their add-ons, and these results tell us we’re on the right track.

In the coming months, we’ll draw on these results to inform our decisions and priorities, to ensure WebExtensions lives up to it promise…and goes beyond.

How an Add-on Played Hero During an Industrial Dilemma

noitA few months ago Noit Saab’s boss at a nanotech firm came to him with a desperate situation. They had just discovered nearly 900 industrial containers held corrupted semiconductor wafers.

This was problematic for a number of reasons. These containers were scattered across various stages of production, and Noit had to figure out precisely where each container was at in the process. If not, certain departments would be wrongly penalized for this very expensive mishap.

It was as much an accounting mess as it was a product catastrophe. To top it off, Noit had three days to sort it all out. In 72 hours the fiscal quarter would end, and well, you know how finance departments and quarterly books go.

Fortunately for Noit—and probably a lot of very nervous production managers—he used a nifty little add-on called iMacros to help with all his web-based automation and sorting tasks. “Without iMacros this would have been impossible,” says Noit. “With the automation, I ran it overnight and the next morning it was all done.”

Nice work, Noit and iMacros! The day—and perhaps a few jobs—were saved.

“I use add-ons daily for everything I do,” says Noit. “I couldn’t live without them.” In addition to authoring a few add-ons himself, like NativeShot (screenshot add-on with an intriguing UI twist), MouseControl (really nice suite of mouse gestures), MailtoWebmails (tool for customizing the default actions of a “mailto:” link), and Profilist (a way to manage multiple profiles that use the same computer, though still in beta), here are some of his favorites…

“I use Telegram for all my chatting,” says Noit. “I’m not a big mobile fan so it’s great to see a desktop service for this.”

Media Keys, because “I always have music playing from a YouTube list, and sometimes I need to pause it, so rather than searching for the right tab, I use a global hotkey.”

“And of course, AdBlock Plus,” concludes Noit.

If you, dear friends, use add-ons in interesting ways and want to share your experience, please email us at editor@mozilla.com with “my story” in the subject line.