Mozilla Add-ons Community Blog
Categories: developers

AMO technical architecture

7 responses

addons.mozilla.org (AMO) has been around for more than 12 years, making it one of the oldest websites at Mozilla. It celebrated its 10th anniversary a couple of years ago, as Wil blogged about.

AMO started as a PHP site that grew and grew as new pieces of functionality were bolted on. In October 2009 the rewrite from PHP to Python began. New features were added, the site grew ever larger, and now a few cracks are starting to appear. These are merely the result of a site that has lots of features and functionality and has been around for a long time.

The site architecture is currently something like below, but please note this simplifies the site and ignores the complexities of AWS, the CDN and other parts of the site.

Basically, all the code is one repository and the main application (a Django app) is responsible for generating everything—from HTML, to emails, to APIs, and it all gets deployed at the same time. There’s a few problems with this:

  • The amount of functionality in the site has caused such a growth in interactions between the features that it is harder and harder to test.
  • Large JavaScript parts of the site have no automated testing.
  • The JavaScript and CSS spill over between different parts of the site, so changes in one regularly break other parts of the site.
  • Not all parts of the site have the same expectation of uptime but are all deployed at the same time.
  • Not all parts of the site have the same requirements for code contributions.

We are moving towards a new model similar to the one used for Firefox Marketplace. Whereas Marketplace built its own front-end framework, we are going to be using React on the front end.

The end result will start look something like this:

image00

A separate version of the site is rendered for the different use cases, for example developers or users. In this case a request comes in hits one of the appropriate front-end stacks. That will render the site using React universal in node.js on the server. It will access the data store by calling the appropriate Python REST APIs.

In this scenario, the legacy Python code will migrate to being a REST API that manages storage, transactions, workflow, permissions and the like. All the front-facing user interface work will be done in React and be independent from each other as much as possible.

It’s not quite micro services, but the breaking of a larger site into smaller independent pieces. The first part of this is happening with the “discovery pane” (accessible at about:addons). This is our first project using this infrastructure, which features a new streamlined way to install add-ons with a new technical architecture to serve it to users.

As we roll out this new architecture we’ll be doing more blog posts, so if you’d like to get involved then join our mailing list or check out our repositories on Github.

7 comments on “AMO technical architecture”

  1. john wrote on

    You probably meant to put the React front-end after nginx, no?

    Out of curiosity, how is RabbitMQ used?

    1. Kumar McMillan wrote on

      RabbitMQ is a message queue that we use to execute background tasks (via Celery). Any operation that is not immediately crucial for rendering the response gets run as a background task for efficiency.

  2. Noitidart wrote on

    Wow that blog post by Wil really brought back memories.

    Yeah to React! What is going to be used for the action/state management (Redux/a flux lib/etc)?

    1. Mark Striemer wrote on

      We’re using redux and currently use redux-async-connect to help with async actions and universal support. You can see what else we’re using at https://github.com/mozilla/addons-frontend.

      1. Noitidart wrote on

        redux-async-connect looks like it offers an awesome set of features compared to redux-thunk, very cool!

  3. André Jaenisch wrote on

    So, what about accessibility? I mean, when React creates _everything_ in its VirtualDOM what about the natural DOM? Will you still have DOM buttons etc.? Or does React cares for adding ARIA attributes?

    Rabimba (https://mozillians.org/de/u/karanjai.moz/) pointed me to https://github.com/reactjs/react-a11y which logs a11y issues to console.

    You should investigate that before migrating the front-end to New Hot Stuff.

    What made you pick React above Angular (2) or Aurelia or similiar?

    1. The amount of functionality in the site has caused such a growth in interactions between the features that it is harder and harder to test.

    “growth in interactions” between who? What features? Or Front-End Back-End interactions? Can you provide an example?

    Why is React better than writing several Django Apps resp. communicate through APIs (thereby decoupling logic)?

    2. Large JavaScript parts of the site have no automated testing.

    Write tests for those?

    3. The JavaScript and CSS spill over between different parts of the site, so changes in one regularly break other parts of the site.

    Introduce a pattern library (CSS) or scope the JavaScript bits better?

    4. Not all parts of the site have the same expectation of uptime but are all deployed at the same time.

    Fix your deployment infrastructure? It does not need to be a single monolith, does it?

    5. Not all parts of the site have the same requirements for code contributions.

    I don’t get this one.

    1. Les Orchard wrote on

      “I mean, when React creates _everything_ in its VirtualDOM what about the natural DOM? Will you still have DOM buttons etc.?”

      TL;DR: React eventually applies changes to the real DOM. It just uses a Virtual DOM to more efficiently plan changes so that a) humans don’t have to do it explicitly with data bindings and such and b) the slower real DOM doesn’t thrash as many possibly redundant operations are performed.

      It goes something like this: Take a snapshot of the DOM as the before picture. Completely rebuild the virtual DOM – which is surprisingly fast and is simple to reason about. Take a snapshot as the after picture. Find the difference between the before and after states, and there you have a minimal set of operations to apply to the real DOM.