Hardware accelerating Firefox

Bas Schouten has posted a lot about hardware accelerating Firefox, specifically with Direct2D and OpenGL. However, there’s been some confusion as to what it is we’re going to hardware accelerate, and how we’re going to do it. This blog post aims to be the definitive reference for hardware acceleration in Firefox.

Please note: We have committed to turning on certain parts of hardware acceleration in developer previews, but that’s not a guarantee of those bits shipping in Firefox 4. We’re going to try hard to ship some form of acceleration, though.

Layers

Layers is a technology that lets conceptually simple, but computationally expensive, parts of the web get offloaded to GPUs. Examples of this include transparency, scaling, composition, and simple animations of those attributes. Since we’re dealing mostly with images on the GPU, we can also accelerate operations on these images, like converting colours; this is especially useful in the case of video, because we can get the GPU to convert from video’s native YCbCr to the GPU’s native RGB, and then scale the result, for example to fullscreen. Both of these operations are expensive on the CPU, but relatively free on the GPU.

The downside of layers is that, in order to get the sort of benefit that’s possible from GPU acceleration, a lot of analysis of the document needs to be done to identify the parts that need to be separated into their own layers. Some of this is relatively easy, like background images; other parts are much harder. Also, layers is designed to accelerate only portions of the web; this means that, in the common case, we will render most of the web page using software, then do only the hardest/slowest part using the GPU directly.

Code currently in mozilla-central supports layers in three modes: a basic software-only mode (using Cairo); a mode that combines software with OpenGL; and a mode that combines software with Direct3D 9 (D3D9). At the time of this writing, only full-screen HTML5 video is rendered using our GPU accelerated (OpenGL and D3D9) layers backends, but in the future we plan to accelerate all of our rendering. To test this hardware acceleration, right-click on a video and select “Full Screen.” GPU accelerated full-screen video is turned on by default; you can check its status in your error console.

Direct2D

Direct2D (D2D) is a Cairo backend that GPU accelerates everything displayed by Firefox. Some parts of the web are more amenable to acceleration than others; for example, moving and scaling images around (like in photos.svg) is much faster. The average page that displays a lot of text and perhaps a few images won’t see nearly as big a speedup, though. The Direct2D Cairo backend doesn’t require any special analysis like the layers backends require, because everything in Firefox is already rendered using Cairo.

The downside to Direct2D is that it’s only supported on Windows Vista + Platform Update and Windows 7, and only then on relatively new GPUs (generally, GPUs that support DirectX 10).

Direct2D support is currently turned off by default, but can be turned on manually using about:config.

26 comments

  1. Would QuartzGL on the Mac help as Direct2D helps on Windows?

  2. Once you’re able to turn on acceleration for all rendering, how will this compare to what the other browser’s known plans are? What is ie9 doing? Chrome? Safari?

    Why is the “layers” approach better than D2D for Windows? Why can’t you get Cairo to support OpenGL in the same way that D2D does?

    Thanks for all the explanations and hard work!

  3. I have an Intel integrated GMA 3100 and Windows XP, so I guess I won’t be joining the fun. It is not that weak a platform, can play Quake Live.

  4. @Jamie:

    IE9 uses/will use Direct2D for hardware acceleration. I have no knowledge of what Chrome and Safari will do in the future, but right now, to the best of my knowledge, they render in software only.

    Layers works on video cards and operating systems that don’t support D2D, like Windows XP. Layers also accelerates some things that Direct2D doesn’t, like video display. That being said, you could (and, in the future, we may) combine Direct2D with a Direct3D 10 layers backend, for the best of both worlds.

    It’s hard to implement a 2D API like Cairo using 3D APIs like OpenGL or Direct3D. That being said, Microsoft did exactly that, and the result was Direct2D. A Cairo OpenGL backend does exist, but it’s not as polished or as fast as Direct2D – yet.

    @John:

    Your video card will be supported using the Direct3D 9 layers backend. It also looks like it’ll support DirectX 10 (and hence Direct2D) if you decide to upgrade to Windows Vista or 7.

  5. “Layers is a technology that lets conceptually simple, but computationally expensive, parts of the web get offloaded to GPUs.”

    Punctuation is confusing here.

    “GPU accelerated full-screen video is turned on by default; you can check its status in your error console.”

    Check it how? I right-clicked a OGV video and viewed it in full screen, but didn’t see anything in the Error Console.

    • @Dan: If you don’t see something in your error console talking about the Layer Manager, you’re not using hardware accelerated layers. We print out information about the driver both for OpenGL and D3D9 when we go into fullscreen.

  6. I checked my configuration the accelerated-layers is true. And I found full-screen video indeed consumes about 10% of my P4 3G CPU, but when the control bar appears, it boots up to 50%-60%, seems like not take the advantage of hardware accelerated composite.

  7. Right now D2D is unusable until this is fixed

    https://bugzilla.mozilla.org/show_bug.cgi?id=554874

    The bug seems to have a patch but no review , pls fix it if you want that we continue testing D2D, because it’s impossible in this state :)

  8. I am glad to hear about the Direct3D 9 layers backend, glad to see an organization support as much users as possible, regardless of how “antiquated” it is.

  9. http://blog.chromium.org/2010/03/introducing-angle-project.html I stumbled upon Angle, wondering if it is developing in parallel with you guys or is it a different goal?

  10. “If you don’t see something in your error console talking about the Layer Manager, you’re not using hardware accelerated layers.”

    OK, I guess the next question is: why not? What can I do to help work out why full-screen video is not being hardware accelerated?

  11. @John:
    Angle is a Google-driven project to implement OpenGL ES on top of Direct3D 9. My understanding is that its primary motivation is to allow the implementation of WebGL on computers that don’t support OpenGL well. Those computers are also the impetus behind Mozilla’s efforts to use Direct3D 9 as a layers backend on Windows.

    @Dan:
    “Why don’t I see hardware acceleration” is sort of complex, but in almost all cases not getting hardware acceleration with full-screen video is a bug. Please file one (in the product Core, component Graphics), along with details about your computer.

  12. Michael Burns

    Is DirectX support meant to be a permanent feature or a stopgap feature until OpenGL improves and it gets better hardware support and what not?

    • Direct3D (and Direct2D) is our permanent plan on Windows, yes. OpenGL is our permanent plan on other platforms. We’ll use whatever’s bes supported on the platform, which on Windows tends to be Direct3D.

  13. Joe Drew: Thanks for your insightful post.

    “That being said, you could (and, in the future, we may) combine Direct2D with a Direct3D 10 layers backend, for the best of both worlds.”

    Does this mean that both approaches would effectively be daisy chained? So Layers analyses the page to split it up into that are hardware accelerated using the D3D layers back-end and other parts which are rendered using Cairo, which in turn is hardware accelerated using the D2D back-end. Correct?

    And if so, isn’t this already possible? Or would it only work with a D3D 10 back-end for layers (and not with the current D3D 9 version)? And if so, is this because D3D 10 (living underneath D2D) cannot be used in combination with D3D 9 (as used by the current layers back-end)?

    In any case, such a “best of both world” scenario makes a lot of sense (at least on Windows), because if the alternative would be to have pref setting that lets users choose between “software rendering”, “Layers (partial D3D/OpenGL acceleration” and “Direct2D acceleration”, confusion among users (even those with above average technical skills) would be major I think :-).

    Regards,

    Matthias

    • @Matthias:

      You’re right about the daisy-chaining. I’m not 100% sure if this is already possible (using the D3D9 layers backend), as I’m not sure of how well Direct2D interacts with DirectX 9 (Direct2D uses DirectX 10 under the covers). It might work, and we should do some experimentation on that!

      Also, you don’t have to worry about prefs regarding hardware acceleration – we’re going to make it Just Work! :)

  14. Frazzled by hours of research and brain throbbing with overload, and not premature seasonal drunkenness, you seem like someone who knows a fair bit about Mozilla and OpenGL.

    In an OpenGL environment (and later maybe with other APIs from the Open… family) I want to composite both XULRunner windows and X11 windows into one image, but under maximum control of Javascript. My thoughts so far are that I’ll need to write an XPCOM component that uses OpenGL and Xlib/XCB libraries to do that, hopefully with Javascript registering event listeners on the component in order to be notified of OpenGL/X11 events. I mention X11 as priority, but a cross-platform solution would be beneficial.

    So, am I proposing a good solution, or am I barking up the wrong tree and missing some interesting tricks hidden in Mozilla’s runtime?

    Regards, Erik

  15. [...] 反正我vdpau硬解时候x还经常崩溃,我可不想一个浏览器让整个x杯具,让内核宕机。如过你敢,我就删。(传说,很多年前,firefox已经实现了linux平台的2d加速 by cairo 1,2,3,至少我这里firefox的gpu跑分没有输给ie9) [...]

  16. [...] playbacks using the new <iframe> embeds will automatically benefit as more and more (and more) modern browsers add hardware acceleration for the native HTML5 <video> [...]

  17. [...] YouTube playbacks using the new iframe embeds will automatically benefit as more and more (and more) modern browsers add hardware acceleration for the native HTML5 video element.Cheers,-Jeff Posnick, [...]

  18. [...] Graphics: GPU-powered browsing. For Firefox 4, we sped up how Firefox draws and composites web pages using the Graphics Processing Unit (GPU) in most modern [...]

  19. [...] YouTube playbacks using the new iframe embeds will automatically benefit as more and more (and more) modern browsers add hardware acceleration for the native HTML5 video element.Cheers,-Jeff Posnick, [...]

  20. [...] example, YouTube playbacks using the new embeds will automatically benefit as more and more (and more) modern browsers add hardware acceleration for the native HTML5 element. Cheers, -Jeff Posnick, [...]

  21. [...] YouTube playbacks regulating a new iframe embeds will automatically advantage as more and more (and more) complicated browsers supplement hardware acceleration for a local HTML5 video element.Cheers,-Jeff [...]

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>