Firefox & Page Load Speed – Part II

blakecutler

23 responses

Let’s start with the punchline: By making a few minor tweaks to our top landing pages, we can drive an additional 60,000,000 Firefox downloads per year.

60 million is a big number, so let’s add a little context. Looking back at our record breaking download day during the launch of Fx3 (June 2008), a total of 8,002,530 people downloaded Firefox. Adding 60 million downloads is akin to running 10 Download Day campaigns per year.

What are the tweaks?

It comes down just one factor — speed. As it stands, our landing pages can be painfully slow. Take a look:

The entire Chrome landing page loads before our header appears. Worse yet, our download button doesn’t appear for 7 seconds! While this video does not reflect the experience of every IE.html visitor, nearly a quarter of our visitors wait at least this long.

Now that we know this page is slow, let’s make it faster. To do so, I thought back to the 2 weeks I shopped a class on high performance web sites. But, I couldn’t remember much, so I asked Ryan Doherty for some tips. He had two simple suggestions: combine (or remove) the JavaScript files and inline the CSS.

Results
After implementing these changes in an A/B test, we saw impressive results. Previously, we predicted a 1 second reduction in page load speed would improve download conversions by 2.7%. In reality, our optimized experimental variation shaved 2.2 seconds off the average page load time and increased download conversions by 15.4%!

With 275,000 daily visitors, a 15% improvement on this single English page translates to 10.28 million additional downloads per year. And, if we’re able to achieve a similar performance boost across our other top landing pages, we’ll drive in excess of 60 million yearly Firefox downloads.

Yeah, but…
15% looks great, but how confident are we?

Running a one-sided Student’s t-test with a means difference of 14%, our experimental data yields a P value of 0.000051. This means that there is only a 0.0051% chance that we would obtain a 14% (or greater) improvement if the real effect wasn’t at least this large.

Next Steps
We’re excited by our initial results, but are only just getting started. We have a bunch more optimizations to try and have some ideas of our own. For example, we want to test a landing page that loads the download button before all other page elements. Later on, we plan to test more elements of Firefox’s adoption funnel, including the download and First Run experiences.

Many thanks go out to Ryan, Fred, Laura, and Matthew, who suggested running performance tests and has already made impressive performance improvements on addons.mozilla.org.

23 responses

  1. kadavy wrote on :

    Wow, very cool! It’s good to hear that speed affects conversion this much. I recently moved some sites from Dreamhost Shared to the Rackspace Cloud, and did some more optimization using YSlow. I know I’ve made my pages load much faster, but I hope I see similar results.

  2. Melvin Ram wrote on :

    Wow those are impressive gains. It’s not really surprising that it would yield better conversion rates but the degree of success is remarkable.

  3. Aaron Peters wrote on :

    Wow, impressive results. Can’t wait to read the next posts in this series!

  4. boloshon wrote on :

    I’m sorry, but do you have a simple way to explain Student’s t-test? I searched for an explaination but failed to understand… 🙂

  5. Vincent Voyer wrote on :

    Hello, the video is nice, it is very similar with http://webpagetest.org videos.

    How did you do to have videos of firefox/chrome/opera page loading ?

    A blog post about how you did your tests would be very appreciated

  6. Steve Dossett wrote on :

    I just sent this to a couple clients as a great example of optimizing an existing site for download conversion rate.

    Thanks for posting.

  7. arstrithis wrote on :

    Some png’s still can be optimized for instance with optipng

  8. Steffen wrote on :

    Have a look at Google’s Page Speed, which is an extension to Firebug. It analyzes a page, provides a score, and makes suggestions, sorted by importance. It even provides links to optimized/minified HTML, JS, Images:
    http://code.google.com/speed/page-speed/

  9. Sean Duran wrote on :

    I’m a huge fan of Firefox. Or, I was. Then I tried Chrome and holy moly, it felt INSANELY fast. I still have to use FF occasionally (for it’s extensions), but when I do it’s a total pain because it’s so slow. I’ve even disabled most of the extensions I don’t use anymore and it’s still slow.

    Being a web developer (among other things), I think the only thing that would make me go back to FF would be if it was faster and it handled HTML5. Playing a YouTube or Vimeo video in HD in FF is choppy. Playback in Chrome is 100% smooth.

    Please, fix FF!

  10. Kevin Suttle wrote on :

    This is great to see Mozilla recognizing and working to improve these issues.

    I second the recommendation to Google’s Page Speed.
    http://code.google.com/speed/articles/

    I also suggest the Yahoo Developer network which shares some of the same info.
    http://developer.yahoo.com/performance/

  11. Corey wrote on :

    Glad to see the increase in downloads. The lower the IE userbase gets the happier I’ll be when clients stop wanting to support IE6.

  12. Ben Griffiths wrote on :

    Wow, it’s amazing how such a small tweak not only improved page load speed so drastically, but what affect that small change had on conversion!

  13. Timothy Lynn wrote on :

    I’m not sure,if speed is important, or not because I’m not fast at any thing, but waiting for pages to load just a little faster might be a good thing less waiting more doing I can deal with it. Mozilla is already the best internet site compare to aol.com , or window’s explorer I have waited hour for them to respond , or load, only a few second for Mozlilla so I,m already convinced

  14. Addy wrote on :

    Unfortunately, I have to agree with one of the previous posters. The people I know who used to sing FireFox’s praises a few years ago all use Google Chrome now.

    It’s significantly faster, doesn’t slow down the same way FF does when extensions are heavily used and it’s threading model just seems to keep things working smoother for longer.

    If I were Mozilla I’d be considering dropping the present rendering engine in favour of a WebKit variant. Your landing pages are unfortunately just one of the areas you guys need to work on speeding up.

  15. Juan wrote on :

    As well as the usual optimizations (using css sprites, combining css and javascript, putting javascript at the bottom, etc), you could consider loading the background image with javascript after all other content was downloaded, like Bing does nowadays in the search result page, so that the navigation menu and download button load first.

  16. Aerik wrote on :

    Excuse the snark, but couldn’t you speed up the landing pages by a ton by only running scripts where they actually serve a purpose? Maybe you shouldn’t have a giant urchin tracking module, two jquery scripts, and that other one loading every single time somebody wants to see what mozilla is all about.

    “Hey, we offer faster speed! Now read about it while we bombard you with four humongous scripts that currently bog you down and ruin your browsing experience just like the rest of the internets.”

    It’s like a car salesmen who puts up signs about how his cars ride smoother, then he puts up speed bumps and cattlegates all over the place.

  17. Patrick Meenan wrote on :

    Hopefully it was a mistake when the data table was put together (or I’m misreading it) but the data table at the end of the article shows a much lower conversion rate for the faster page (variation) with the control page having the higher conversion rate.

  18. Blake wrote on :

    @Patrick – good catch! We’ve fixed the error.

  19. Jon wrote on :

    We are constantly preaching to people to make sure they tests their sites constantly. It is critical, as you only have 50 milliseconds. That’s for the detailed info. Also, I would have to agree with Addy on firefox overall slowdown, and add that crashes are also way up. It’s a shame, people are flocking to Chrome.

  20. Stephen Wille wrote on :

    “He had two simple suggestions: combine (or remove) the JavaScript files and inline the CSS.”

    do your really mean to move the CSS from it’s own file into the HTML code, or is that a typo?

    i can see either being the case; extra files mean extra http requests, but larger files mean longer load times, or so i’ve been lead to believe. so, which is it? did you really move the CSS into the HTML code?

  21. Shailesh Tripathi wrote on :

    Firefox is my favorite browser. It helps me for secure and fast browsing.

  22. web design company wrote on :

    Really good stuff —I’ve never pulled designs into PS for color layering, but will definitely be trying that. One thing I like to do if I’m stumped on a particular part of a page I’m wireframing is to photocopy the partially completed design a few times so it’s easier to iterate on a section without having to redraw the rest over and over.

  23. web design company wrote on :

    PageSpeed generates its results based on the state of the page at the time you run the tool. To ensure the most accurate results, you should wait until the page finishes loading before running Page Speed. Otherwise, PageSpeed may not be able to fully analyze resources that haven’t finished downloading.