26
Apr 11

Introducing the Azure project

Motivation

Firefox 4’s graphics performance is great, and some last-minute “low-hanging fruit” inefficiencies Robert O’Callahan found made us even faster on some demos in Firefox 5. We’re not content with “great”, though, and our investigations into how to make drawing even faster have revealed that some of our choices in Gecko’s graphics engine aren’t optimal for performance.

How Mozilla renders

To display a web page, Gecko’s layout engine looks at how that page is constructed, and divides it into independent layers. Each of those layers is drawn using Cairo, a 2D graphics API built on top of the operating system’s graphics APIs. Cairo’s drawing into layers can be hardware accelerated — this is known as “content acceleration” — or done in software. Right now, content acceleration is fastest on Windows Vista and 7, where we use Direct2D, but we want to be able to extend good acceleration to all operating systems, including mobile.

After drawing, layers are composited together using either the platform’s 3D APIs (OpenGL or Direct3D) or Cairo, and the result is displayed on the screen.

Format conversions

Cairo uses a fixed point representation of numbers. Unfortunately, this means that when Cairo calls get mapped to an API like Direct2D, which uses floating point numbers, two conversions are required: one from from your internal floating point representation to Cairo’s fixed point representation, in which it is initially stored, and then back to floating point again when Cairo makes Direct2D calls.

Statefulness

Cairo is a stateful API. This means that any given function call depends on the results of the previous function calls to determine its result. Unfortunately, statefulness strictly adds overhead to an intermediate graphics API: if it maps to another stateful API, Cairo has to track its own state separately from the underlying API (usually due to subtle differences between the two). Also, Cairo uses a stateless surface API internally; whether the underlying API is stateful or stateless, there’s always a conversion from one to the other. Further, the HTML5 Canvas element is a stateful API that’s built on top of Cairo, and it has to have its own state tracking too.

What makes this worse is that Gecko is better-suited by a stateless API; when we begin rendering any part of a web page, we need to reset most of Cairo’s stored attributes, since they’re determined by the element’s CSS.

What can we do about it?

We would very much like to remove these unnecessary conversions and state tracking. We’d also like to be able to hardware accelerate content drawing in an easy and cross-platform way.

Mozilla needs a stateless graphics API that’s closer both to platform APIs and hardware.

This API needs to be close enough to Direct2D to be little more than a thin wrapper, but general enough to be implemented on OpenGL and Direct3D 9.

The Azure project

To accomplish this, the Mozilla graphics team is creating a new 2D graphics API. It’s going to be stateless, and significantly closer to Direct2D. We’re also going to develop new “3D” backends for drawing content in an accelerated way to OpenGL, Direct3D 9, and Direct3D 10.

The backends

Once complete, the new 2D API will be implemented by several different backends, depending on OS, hardware, and driver. We will also have a fallback implementation, using Cairo, that we’ll use when we don’t have a more specialized backend available.

Direct2D

We don’t make the best use of Direct2D right now. Part of it is the floating/fixed point impedance mismatch I described earlier, but part of it is capabilities Cairo exposes that Direct2D doesn’t implement. These need to be worked around or emulated using software. The API we create as part of the Azure project will be close enough to Direct2D to not require fallbacks.

Quartz / CoreGraphics

On OS X, Cairo uses its Quartz backend for all drawing. Quartz can be fast: Safari is faster than Firefox on some demos, even though we both use the same backend for drawing. We believe one reason this is the case is because Quartz is stateful: Cairo needs to convert from its stateful API to its internal stateless surface API, then back to Quartz’s stateful API. We’ve designed Azure’s stateless 2D API so that it can map from stateless to stateful in a more optimal, and optimizable, way.

Cairo

Cairo is our failsafe implementation. It’s the backend we’ll use before others are ready or when we explicitly choose not to use hardware acceleration, for example because of driver issues. We also need Cairo for printing support.

It’s possible that we will decide to modify our internal copy of Cairo to allow us to use the internal stateless surface API directly, since we think Cairo’s surface API is a pretty good match to Azure’s 2D API.

3D Backends

The goal of the “3D” backends is to implement Azure’s 2D API on top of the common 3D APIs (OpenGL and Direct3D) so we can have content acceleration on Windows XP, OS X, Linux, and mobile. Our intent with these 3D backends is to be fully cross-platform, including Windows 7. This lets us better control our performance characteristics and bugs, rather than spending a lot of time and energy working around Direct2D bugs.

Electrolysis and Azure

The Azure 2D API will make it possible for us to hardware accelerate content drawing, even in the distant future when we “sandbox” our content processes. This new 2D API is required for hardware accelerated sandboxed content processes, but isn’t required for the initial Electrolysis roll-out.

Timeline and checkpoints

Over the past few weeks, we created a proof-of-concept Direct2D backend to test our hypothesis that we could create a faster non-Cairo API. Our hypothesis was correct — our backend is significantly faster than the Direct2D Cairo backend on some commonly-used Canvas demos and benchmarks. This work is all available in the graphics project branch.

Our initial success led to our current plan: to design and test this new 2D API as a backend for the HTML5 canvas element. The graphics team has included getting the Azure Direct2D canvas backend enabled in mozilla-central as a Q2 goal.

We are also in the middle of writing similar canvas backends for Cairo and Quartz, both of which are already in the graphics project branch, but those are not yet operational. We haven’t defined specific time goals for those backends.

Postscript: Why “Azure”?

Jim Clark invented the first GPU at Stanford in 1979, called the “Geometry Engine”. He founded SGI based on this work.

SGI made many workstations and servers with “colour” names like Onyx, Indigo, and Crimson. They are also the original inventors of OpenGL.

After SGI, Clark founded a little-known software company called Netscape.

It seemed only fitting to pay homage to him in this small way!