Firefox Accounts, Sync 1.5, and Self Hosting

Firefox Sync offers best-in-class security and privacy: Mozilla’s servers never see your raw password, and all of your Sync data is encrypted locally, before it leaves your computer. However, some users want to go one step further and host all of their Sync data themselves. The ability to self-host is central to Mozilla’s mission, as it ensures that the convenience of Sync doesn’t come with hidden costs like strict lock-in.

With Sync 1.1, self-hosting was as easy as setting up ownCloud, using one of several Python modules, or for the more intrepid users, running the same code that Mozilla used in production. OpenBSD even includes a Sync 1.1 server in its ports tree!

With Sync 1.5’s recent release, we’re back to square one: all of Mozilla’s code is open source, and people are already running their own Sync 1.5 stacks, but it will take a few months before easier-to-use, self-contained projects emerge.

Of course, if you’re currently using using a self-hosted Sync 1.1 server, Firefox will continue to work with it for several months while we sort out self-hosting of Sync 1.5.

The Relationship Between Sync and Firefox Accounts

One challenge with self-hosting Sync 1.5 is understanding the relationship between Sync 1.5 and Firefox Accounts (FxA). Indeed, this is a frequent topic in the #sync IRC channel.

The Firefox Accounts system serves two purposes:

  1. It is a single, common authentication system for Mozilla services like Sync, the Firefox Marketplace, and Firefox OS’s find-my-device feature.
  2. It stores a password-wrapped version of your Sync key, so that you (and only you) can recover your data, even if you lose all of your devices.

By keeping authentication in one well-defined place, and sharing that infrastructure between projects, we’re able to dramatically reduce complexity and maintenance costs associated with running many services. Sync can focus on Sync, while letting Firefox Accounts handle accounts.

Unfortunately, this happens to increase complexity if all you care about is self-hosting a single service, like Sync.

The Moving Parts

Right now, the Firefox Accounts system is mostly comprised of a database and three servers written in Node.JS:

  1. fxa-auth-server, which handles the Firefox Accounts backend and API, including provisioning identity certificates for users.
  2. fxa-content-server, which serves up static content, like the UI that you see when you visit about:accounts in Firefox.
  3. browserid-verifier, which makes it easy for services like Sync to verify identity certificates issued by the auth server.

Sync has a database and two servers written in Python:

  1. tokenserver, which handles incoming auth requests, issues long-lived bearer tokens, and directs the browser to an appropriate storage server.
  2. syncstorage, which accepts bearer tokens and allows the browser to store and retrieve encrypted Sync data.

Sync and FxA are decoupled, which leads to two interesting self-hosting options:

Method 1: Just the Data

It’s possible to host just the Sync components, while still relying on Mozilla’s hosted Firefox Accounts infrastructure for authentication.

With this method, you still store all of your encrypted Sync data on your own server, but you use Mozilla’s servers for authentication and for backing up a password-wrapped version of your Sync key.

Because Sync uses client-side (PBKDF2 + HKDF) key stretching, Mozilla never learns your plaintext password, and thus cannot unwrap your Sync key. Additional server-side stretching (scrypt + HKDF) further protects against brute-force attacks if the Firefox Accounts database is ever compromised.

Method 2: Full Stack

Alternatively, you could host both the Sync and Firefox Accounts components, completely removing any dependence on Mozilla for Sync.

Self-hosting is limited to browser-based services like Sync. Web sites, like the Firefox Marketplace, may require an additional account with the centralized Firefox Accounts server before allowing you to log in.

The Plan

Right now, we’re focusing on getting Method 1 working as well and as quickly as possible. Most of that work is happening in the syncserver repo, which is a Python project that combines both the tokenserver and syncstorage projects into a single package.

We also need to resolve Bug 1003877, which is currently preventing either method from working on Firefox for Android.

Lastly, we need to ensure that the work we do to facilitate self-hosted Sync is the right work. Namely, how do we go from “technically possible” to “reasonably easy” for advanced users? To get this right, we’ll need to draw on the experience of projects like ownCloud, which is at the forefront of making Sync 1.1 easy to self-host, as well as users who have self-hosted in the past or who are interested in self-hosting now.

If you want to roll up your sleeves and get involved, please join the sync-dev mailing list and introduce yourself in #sync on irc.mozilla.org!

2 responses

  1. blissdev wrote on :

    This is great news, glad to hear progress is being made on this front.

  2. allo wrote on :

    Some experience:
    I am using Sync 1.1 with https://github.com/jedie/django-sync-server for quite a while now. This is a nice “it just works” solution, with very little overhead. Because i do not host many accounts, i just use a sqlite backend, which works. The django admin interface is okay to manage users (i.e. delete old ones).

    The current building blocks:
    I set up a chroot and started installing was described in https://docs.services.mozilla.com/howtos/run-sync-1.5.html
    The syncserver was running after just a minute quite nice, i put it behind a reverse proxy setting the external url accordingly.

    Then i wanted to continue by setting up the fxa-Stuff, as i would like to have the full stack solution.
    This has quite a few problems:
    – node.js.
    — There is no node.js in debian stable (solved for the moment by using a debian jessie chroot)
    — node.js is a bit of overhead. While i am already running a lot of django projects, i do not need node.js for anything, yet.
    — npm install is a nightmare. I do not know if it’s node.js or you project, but it seems that its installing half an operation system, getting hundreds of packages. Some are broken and expect the “nodejs” binary to have the name “node” (which needs a symlink with debian jessie packages)
    — the config.js is a nightmare, too. Put the verbose js syntax aside (why not a config syntax like yaml?), it is a loooong default config, which seems to include a lot of stuff, i do not care about when i just want the defaults.
    — the config.js defines three databases: mysql, memory and httpdb. I do not really know what’s httpdb, but what i am really missing there is sqlite. I guess its possible to use it, but its not “just put sqlite here and the path to the db there”. And i guess people with bigger servers would like to see postgres.

    As i did not finish setting up the two fxa servers (yet), i do not know if it would work.

    Another thing which seems to be missing from the howto: the tokenserver. The howto writes about two needed servers.

    But to say something nice in the end: from the first time i asked in the IRC and looked at the docs, the documentation improved a lot. I am looking forward to a nice “setup a minimal full-stack sync” howto :).