Improving Themes in Firefox

Last year, we started thinking about how to improve themes in Firefox. We asked theme designers and developers for their thoughts too, and received over 250 responses. After sorting through the responses and a few months of deliberation and experimentation, we have a plan, and would like to share what we’re going to do and why.

Let’s start with the “why.”

Currently, Firefox supports two types of themes. Complete Themes and Lightweight Themes (aka LWTs, formerly “Personas”, and now just “Themes” on addons.mozilla.org).

Lightweight Themes are very popular. There are currently over 400,000 such themes on addons.mozilla.org (AMO). They’re extremely simple to create; anyone can create one by just uploading an image to AMO. I made my first LWT eight years ago, and it still works fine today without any changes. However, LWTs are very limited in what they can do. They can lightly modify the default Firefox appearance with a background image and set a couple of colors, but nothing else. The survey confirmed that there’s a lot of interest in giving LWTs more creative control over how the browser looks. More backgrounds, colors, and icons on more UI elements.

Complete Themes, the second type of theme, completely replace the default Firefox appearance. Authors must provide all the needed CSS, images, and icons for the entire browser (from scratch!), which makes Complete Themes flexible but difficult to create and maintain. Authors have to understand Firefox’s complex and ever-changing UI internals, and there’s little documentation beyond Firefox’s source code. This leads to a serious compatibility burden, as authors need to invest time to keep their theme working with each Firefox release – only 60 of the 500 Complete Themes on AMO are compatible with current Firefox releases. Further, we’re not able to directly fix these issues without limiting our ability to improve Firefox. And since only 0.089% of Firefox users use a Complete Theme (less than 4% of the usage of LWTs), we’re going to focus on improving theming in other ways.

Firefox extensions (as opposed to themes) can also run into these problems, as there is no JavaScript API to control the appearance of the browser. Legacy add-ons wanting to do this had to directly alter the UI’s internal DOM and CSS. As with Complete Themes, this can give an extension great power, but it comes with a serious price in complexity and compatibility. WebExtensions are unable to access any UI internals, and so are particularly in need of a solution.

What we’re doing

We want to fix Firefox so theming is better for everyone.

The next generation of Firefox themes will blend LWTs’ ease of authoring with the additional capabilities we see most often used in Complete Themes. At its core is a JSON manifest, mapping defined property names to the underlying UI elements. Theme developers will be able to control a variety of styles on these properties (such as colors, icons, and background images), and Firefox will ensure the manifests are supported in a stable and well-documented way across future releases and UI updates. These themes will be layered on top of the default Firefox appearance, so you can create a trivial theme that just changes one property, or a complex theme that changes all of them.

The goal is to provide capabilities that enable people to make themes they love and use. We think it’s possible to make themes in Firefox powerful without the previous pitfalls. To get started, we’ll initially support the same properties as Chrome, to make the thousands of Chrome themes more easily available in Firefox. Then we’ll expand the set of properties in our API, so that Firefox themes will be able to do more. We expect to continue adding to this theming framework over time, with your feedback helping to guide what’s needed.

We also recognize that a property manifest won’t have all the capabilities of direct CSS manipulation, especially during the time we’re expanding coverage to cover the most common use cases. So, for theme authors who need additional capabilities (and are willing to bear the burden of supporting them), we’ll provide an “experimental” section of the manifest to continue to allow direct CSS manipulation of the Firefox UI. This is similar in spirit to WebExtension Experiments, and usage will also be restricted to pre-release versions of Firefox.

A WebExtensions API

Finally, we’re adding a WebExtensions API for theming. It will have the same capabilities as the JSON manifest’s properties, except via a JavaScript API, so add-ons can make dynamic changes at runtime. This will enable add-ons to do things like adjusting colors based on the time of day (e.g. Flux), or matching your theme with the weather outside.

Questions and Feedback

For more information on our current work, see our Engineering Plan. We’ve just started some of the foundational work, but would welcome your input as we move towards building the first version that will be usable by theme authors. Our goal is to have this in place before Firefox 57 ships in November, which will end support for Complete Themes. We currently expect early testing to begin in Nightly over the next few months.

Please address your questions and feedback to the Dev-Addons list at dev-addons@mozilla.org.

We’re also keeping a list of Frequently Asked Questions, check it out to see if it answers your question.

The Road to Firefox 57 – Compatibility Milestones

Back in November, we laid out our plans for add-ons in 2017. Notably, we defined Firefox 57 as the first release where only WebExtensions will be supported. In parallel, the deployment of Multiprocess Firefox (also known as e10s) continues, with many users already benefiting from the performance and stability gains. There is a lot going on and we want you to know what to expect, so here is an update on the upcoming compatibility milestones.

We’ve been working on setting out a simple path forward, minimizing the compatibility hurdles along the way, so you can focus on migrating your add-ons to WebExtensions.

Legacy add-ons

By legacy add-ons, we’re referring to:

Language packs, dictionaries, OpenSearch providers, lightweight themes, and add-ons that only support Thunderbird or SeaMonkey aren’t considered legacy.

Firefox 53, April 18th release

  • Firefox will run in multiprocess mode by default for all users, with some exceptions. If your add-on has the multiprocessCompatible flag set to false, Firefox will run in single process mode if the add-on is enabled.
  • Add-ons that are reported and confirmed as incompatible with Multiprocess Firefox (and don’t have the flag set to false) will be marked as incompatible and disabled in Firefox.
  • Add-ons will only be able to load binaries using the Native Messaging API.
  • No new legacy add-ons will be accepted on addons.mozilla.org (AMO). Updates to existing legacy add-ons will still be accepted.

Firefox 54-56

  • Legacy add-ons that work with Multiprocess Firefox in 53 may still run into compatibility issues due to followup work:
    • Multiple content processes is being launched in Firefox 55. This enables multiple content processes, instead of the single content process currently used.
    • Sandboxing will be launched in Firefox 54. Additional security restrictions will prevent certain forms of file access from content processes.

Firefox 57, November 14th release

  • Firefox will only run WebExtensions.
  • AMO will continue to support listing and updating legacy add-ons after the release of 57 in order to have an easier transition. The exact cut-off time for this support hasn’t been determined yet.
  • Multiprocess compatibility shims are removed from Firefox. This doesn’t affect WebExtensions, but it’s one of the reasons went with this timeline.

For all milestones, keep in mind that Firefox is released using a “train” model, where Beta, Developer Edition, and Nightly correspond to the future 3 releases. You should expect users of pre-release versions to be impacted by these changes earlier than their final release dates. The Release Calendar lists future release dates per channel.

We are committed to this timeline, and will work hard to make it happen. We urge all developers to look into WebExtensions and port their add-ons as soon as possible. If you think your add-on can’t be ported due to missing APIs, here’s how you can let us know.

Add-ons Update – 2017/02

Here’s the state of the add-ons world this month.

If you haven’t read Add-ons in 2017, I suggest that you do. It lays out the high-level plan for add-ons this year.

The Review Queues

In the past month, 1,670 listed add-on submissions were reviewed:

  • 1148 (69%) were reviewed in fewer than 5 days.
  • 106 (6%) were reviewed between 5 and 10 days.
  • 416 (25%) were reviewed after more than 10 days.

There are 467 listed add-ons awaiting review.

If you’re an add-on developer and are looking for contribution opportunities, please consider joining us. Add-on reviewers are critical for our success, and can earn cool gear for their work. Visit our wiki page for more information.

Compatibility

The blog post for 52 is up and the bulk validation was already run. Firefox 53 is coming up.

Multiprocess Firefox is enabled for some users, and will be deployed for all users very soon. Make sure you’ve tested your add-on and either use WebExtensions or set the multiprocess compatible flag in your add-on manifest.

As always, we recommend that you test your add-ons on Beta and Firefox Developer Edition to make sure that they continue to work correctly. End users can install the Add-on Compatibility Reporter to identify and report any add-ons that aren’t working anymore.

Recognition

We would like to thank the following people for their recent contributions to the add-ons world:

  • eight04
  • Aayush Sanghavi
  • zombie
  • Doug Thayer
  • ingoe
  • totaki
  • Piotr Drąg
  • ZenanZha
  • Joseph Frazier
  • Revanth47

You can read more about their work in our recognition page.

web-ext 1.8 released

A new version of web-ext has been released! Web-ext is the recommended tool for developing WebExtensions on Firefox, because it has the ability to automatically reload your WebExtension as you make changes.

Since our last blog post, version 1.7 and 1.8 have been released. The full change log is on github.

The run command now shows a desktop notification if auto-reloading results in an error: image01

Other options added to the run command include:

  • Addition of a –start-url option. This will start Firefox at a particular URL and assists in testing.
  • Addition of a –browser-console option. This will open the Browser Console by default, which is where any errors or logging will be shown.
  • Addition of –pref option. This will load the specified preferences into Firefox. For example: –pref privacy.userContext.enabled=true
  • When a reload occurs, it will show you the last reload time more concisely.

An –ignore-files option was added, so by default the web-ext-artifacts directory is added to that list when building your extension.

A new option to linting, –warnings-as-errors, will allow you to make the linter more strict, so that warnings are raised as errors. Also, when you run web-ext and you have an error in your JSON, you’ll get an error message showing the line number. As an example:

image00

Any command run will let you check to see if a new version of web-ext exists, ensuring that you are using the latest version of web-ext.

Finally a regression on Windows was fixed, but more importantly the test suite was enabled on Windows to reduce regressions on Windows in the future.

Special thanks to all the people who contributed to this release: Aniket Kudale, Jostein Kjønigsen and eight04. A special thanks to Elvina Valieva and Shubsheka Jalan who have been contributing via the Outreachy program.

New Developer Hub landing and listing pages

With last week’s product update, we introduced a new look for the main and top-level listing pages of the Developer Hub. We went for a modern and friendly design, and surfaced more useful information and links.

The topmost button you see before you sign in takes you to documentation on how to build an add-on. Moving down the page, you see links to submit and manage your add-on, and information on porting an extension.

The lower sections include a feed of our latest blog posts, validation and compatibility tools, and a much more organized and informational footer. We also added a new section on how to contribute to the add-on project.

We pinned commonly accessed links to the top and added an announcement area for people who are signed in. The list of your add-ons is easier to read, with pertinent information like Last Updated and Status more prominently displayed. It’s also possible to click to submit a new theme directly from this page (previously, it only appeared on theme pages).

signedindevhub

You’ll see this new design on Android as well, although only the signed-out view is mobile-friendly for now.

We are making steady improvements to the add-on submission and management experience throughout the year, and hope you enjoy this latest update.

February’s Featured Add-ons

Firefox Logo on blue background

Pick of the Month: Popup Blocker

by Jeremy Schomery
Not only blocks every popup you encounter, but offers an easy way to preview the URL and decide for yourself if you want to open the page.

“I tried all the popup blockers, but this works best for me, and the developer is updating it.”

Featured: Skype™ Web

by Morni Colhkher
Access Skype Web right from your toolbar menu.

“This extension allows me to use Skype while doing miscellaneous internet things, and also notifies me whenever I receive a new message.”

Featured: Desktop Messenger for WhatsApp™

by Elen Norphen
Bring WhatsApp messaging, alerts, and more right into the browser.

“Wow! Awesome add-on! Now I can download pictures from my messages onto my PC.”

Featured: AdBlocker Ultimate

by AdBlocker Ultimate
This add-on pretty much blocks it all—tracking, malware, and of course ads (and no, there’s no whitelisting).

“Everyone should have this.”

Featured: Turn Off the Lights

by Stefan VD
Great for watching video content when you want to really immerse yourself in the media, Turn Off the Lights fades away everything in the video’s periphery.

“So useful. Works as advertised, small memory footprint. Never breaks or interferes with web pages.”

Nominate your favorite add-ons

Featured add-ons are selected by a community board made up of add-on developers, users, and fans. Board members change every six months. Here’s further information on AMO’s featured content policies.

If you’d like to nominate an add-on for featuring, please send it to amo-featured@mozilla.org for the board’s consideration. We welcome you to submit your own add-on!

Mixing Listed and Unlisted Add-ons on addons.mozilla.org

Firefox add-on developers are free to distribute their add-ons as they see fit, as long as they aren’t forced on users. This comes down to two distribution methods: list it on addons.mozilla.org (AMO), or distribute it themselves via a Web page or application installer.

The implementation of add-on signing on AMO added some unwanted restrictions to these options, essentially making developers choose one distribution channel or the other for all versions of an add-on. Switching between one channel and the other is possible, but complicated.

Today we’re removing these restrictions and enabling mixed listed and unlisted versions for add-ons on AMO. When submitting a new version of an add-on, developers will be able to choose if they want to host it on AMO or on their own. This makes it possible to quickly sign an add-on file for user-testing, create a self-distributed pre-release channel, and more. The only limitation is that version numbers need to be unique across both channels.

Part of this feature has been active on AMO for a while. You may have noticed the new add-on and version submission flows, which were the first set of changes we pushed out.

New add-on submissionToday we’ll enable the distribution channel choice for new version uploads, as well as changes to the Developer Hub to make it easier to manage mixed versions.

New version submission

This is the result of many months of engineering work. The add-on lifecycle, signing, reviews, and various add-on/version/file status combinations are at the core of AMO and are surprisingly complex. I’d like to thank Andrew Williamson and Mathieu Pillard for taking on the bulk of this monumental task, and Bram Pitoyo for doing the UX work.

Please report any AMO issues on GitHub.

WebExtensions in Firefox 53

Firefox 53 landed in Developer Edition this week, so we have another update on WebExtensions for you. With the latest release, a slew of new APIs are now available to help legacy add-on developers transition and extension developers port from other browsers.

New APIs

The majority of browser.browsingData API was implemented. This API allows you to delete data from Firefox that the user has accumulated while browsing. This includes data stored in the following places: plugin data, form data, history, cookies, downloads, passwords, service workers and the cache.

Parts of the browser.identity API was implemented. This makes it easier for extensions to integrate with OAuth providers. The getRedirectURL and launchWebAuthFlow methods have been implemented, but the areas related to Google Accounts have not been implemented.

As previously mentioned, the browser.storage.sync API had been in a testing phase. It’s passed testing now, and is turned on by default. As this feature rolls out to our users, we will continue to do more testing. It’s worth noting that the storage service is not intended for large amounts of data and comes with no guarantees around stability or uptime.

A new API, browser.contextualIdentities, landed in Firefox 53 to support the security container feature. In Firefox 52, support for contextualIdentities was added to tab and cookie stores. This API provides access to query existing identities, create, update and remove those identities.

As an example:

browser.contextualIdentities.query({})
  .then((result) => {
    for (let identity of result) {
       console.log(identity);
    }
});

Outputs the existing identities:

Object { name: "Personal", icon: "fingerprint", color: "blue", cookieStoreId: "firefox-container-1" }
Object { name: "Work", icon: "briefcase", color: "orange", cookieStoreId: "firefox-container-2" }
..[snip]

This API is behind the same browser preference, privacy.userContext.enabled, that the rest of the contextual identity code is. We expect the API to track that preference for the moment.

Work has begun on the browser.devtools API and a major foundation of this landed in Firefox 54. We are hoping to land the remained of devtools in Firefox 54, which will allow many developer focused add-ons to work.

API Changes

Context menus have had a big improvement. They can now be applied to pageActions, browserActions, password inputs and tabs.

Screenshot 2017-01-23 14.59.01
A context menu item on a tab

A small change has been made to context menus so they now inherit the submenu contexts from their parent by default.

There was a previous issue where the requestBody on webRequest was not available on release versions of Firefox due to concerns about performance. Those issues have now been resolved and this functionality will be available in release from Firefox 53 onwards.

There was a significant increase in performance for browser.tabs.query which will speed up queries when a large number of tabs exist. Also in tabs, the onUpdated event will now fire when the title of a tab changes.

To complete the browser.sessions API, the browser.sessions.onChanged event landed. This allows extensions to tell when recently closed tabs or windows are changed.

You can now insert CSS into Firefox as a user sheet. As an example:

browser.tabs.insertCSS({..., cssOrigin: "user'})

Finally, function keys now work in commands.

Permissions

With Firefox 53, required permissions have been enabled for WebExtensions add-ons. The permissions dialog is behind a preference while we complete QA on the feature. We hope that permissions will be turned on by default for Firefox 54. To activate permissions, please create the preference: extensions.webextPermissionPrompts as a boolean and set it to true.

When installing an add-on, a user will get a prompt like this:

Screenshot 2017-01-23 16.20.16
Installing an add-on

Updates will proceed normally, unless you update the permissions of your add-on. In that case  the add-on update will be staged. Unlike Chrome, the existing add-on will continue to work and will not be disabled. Firefox users will get a notification in the hamburger menu:

Screenshot 2017-01-23 17.23.12
An update that includes permission changes to an add-on.

And when they click on it, they will see a new permissions dialog outlining the changes. In this example, it shows that I have added the permission nativeMessaging to the add-on:

Screenshot 2017-01-23 17.23.02
The permission prompt on an update

Finally, if your add-on is being sideloaded, the notification will also change to a new flow. An item in the hamburger menu is shown (similar to above), followed by a slightly different permission prompt:

Screenshot 2017-01-23 17.39.55
A sideloaded extension

This is a big feature and many details are currently being finalized. So feedback or bugs are encouraged so we can solve any problems before Firefox 54.

New contributors

A lot of contributors helped out on this release, so a big thank you to them! They are: Srivatsav Gunisetty, Laurent, André Bargull, Rob Wu and Tomislav Jovanovic.

Preventing Add-ons & Third-party Software From Loading DLLs Into Firefox

Updated 2017-01-25: Thanks for the feedback given on this post. To lessen the impact of this change on developers, we’ve clarified our policy and updated this post such that it applies only to new add-ons, and those that do not rely on Firefox internals. It also includes further clarification that add-ons loading binaries that depend on Firefox internals could be subject to blocklisting starting in Firefox 53.

Loading a DLL or binary component into the Firefox process is a method employed by third-party software to enable low-level interactions between Firefox and the operating environment and/or applications that run within it. Binaries can be loaded via an add-on using JS-ctypes, or injected directly into the process using other methods to enable functionality not available natively. These techniques, however useful to the developer, do not always account for underlying changes to Firefox, and are frequently the root cause of startup crashes for users running a new version of Firefox for the first time.

Over the past year, these startup crashes have resulted in the delay or revision of four Firefox releases. The crashes erode confidence in Firefox, and can render Firefox and the information it contains unusable. Users of Firefox need the confidence that their browser won’t crash on an update, and that they’ll have a positive experience using the newest and most secure version available. Firefox release managers need the confidence that they can release new versions of Firefox without worrying about crashes brought on by third-party software.

With the introduction of the Native Messaging API in WebExtensions in Firefox 50—released on November 8, 2016—extensions are able to communicate directly with a host executable running in a separate process. These executables are installed separately, and provide low-level interactions outside of the Firefox process without the possibility of crashing it. The use of Native Messaging with extensions is the supported method for third-party developers to perform interactions that are not available natively, and other methods, including using JS-Ctypes, are actively discouraged.

Starting with Firefox 53, to be released on April 18, Mozilla will introduce changes designed to discourage and/or prevent the loading of third-party binaries into the Firefox process(es) that include:

  • Updating our add-on policies and validation methods to reject:
    • Any new add-ons that load binaries using JS-ctypes or other methods
    • Any add-on that loads libraries that depend on Firefox internals.
  • Updating the blocklisting policy to include:
    • Blocking of libraries that third-party software loads (or attempts to load) DLLs into the Firefox process(es) using any method
    • Blocking of add-ons that incorporate binaries that depend on any internal of Firefox
  • Product changes to better protect Firefox from DLL injection

These changes will also prepare us for wider adoption of the 64-bit version of Firefox on Windows in the near future, as some existing DLLs that are injected or loaded will not be compatible.

Add-on developers who are currently using JS-ctypes should begin immediately transitioning to the Native Messaging API using WebExtensions. Documentation and examples for Native Messaging are available on MDN, and you can ask questions or share concerns about these changes in these communication channels.

Migrating to WebExtensions: port your stored data

WebExtensions are the new standard for add-on development in Firefox, and will be the only supported type of extension in release versions of Firefox later this year. Starting in Firefox 57, which is scheduled to arrive in November 2017, extensions other than WebExtensions will not load, and developers should be preparing to migrate their legacy extensions to WebExtensions.

If you have a legacy extension that writes data to the filesystem, and you’re planning to port it to WebExtensions, Embedded WebExtensions are available now in Firefox 51 to help you transition. Embedded WebExtensions can be used to transfer the stored data of your add-on to a format that can be used by WebExtensions. This is essential because it lets you to convert your users without the need for them to take any actions.

What is an Embedded WebExtension?

An Embedded WebExtension is an extension that combines two types of extensions in one, by incorporating a WebExtension inside of a bootstrapped or SDK extension.

Why use an Embedded WebExtension?

There are attributes (functions) of legacy add-ons that are used to store information related to the add-on that are not available in WebExtensions. Examples of these functions include user preferences, arbitrary file system access for storing assets, configuration information, stateful information, and others. If your add-on makes use of functionality like these to store information, you can use an Embedded WebExtension to access your legacy add-on data and move it over to a WebExtension. The earlier you do this, the more likely all your users will transition over smoothly.

It’s important to emphasize that Embedded WebExtensions are intended to be a transition tool, and will not be supported past Firefox 57. They should not be used for add-ons that are not expected to transition to WebExtensions.

How do I define an Embedded WebExtension?

To get started, read the documentation below. You can also contact us—we’re here to help you through the transition.

MDN docs: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Embedded_WebExtensions

Examples: https://github.com/mdn/webextensions-examples/tree/master/embedded-webextension-bootstrapped

https://github.com/mdn/webextensions-examples/tree/master/embedded-webextension-sdk