Do Ads Driving Firefox Downloads Affect Firefox Downloads?

We’ve previously talked extensively about one of Mozilla’s core marketing tactics – search engine marketing.  One of the questions we always come back to is: “what macro level effect does this activity have?”  In other words, we know this channel is driving clicks and Firefox downloads every day, but is this effort actually aiding Firefox adoption in some way?

This thought might seem anti-intuitive.  If these marketing campaigns are driving a mass number of clicks and downloads, of course it *seems* as though we’re having some small impact on Firefox adoption.

To test these questions, we recently set-up an experiment much like an experiment we conducted last fall.  Here’s what we did:

  • Starting in July, we alternated our bidding on alternate days on a major search engine over a five week period
  • Some weeks we turned off our branded search campaigns on Mondays and Wednesdays (to compare with Tues/Thurs during those weeks), and during some weeks we turned off our branded search campaigns on Tuesdays and Thursdays (to compare with Mon/Wed within those weeks)
  • For the latter example, a Monday/Wednesday combination with our campaigns left active would have benefited from about 80,000 ad clicks that the Tuesday/Thursday period within that week did not benefit from

Below are the findings when looking at Firefox download numbers on a macro-level, i.e., across all channels, locales, etc.  This is a very large macro-level number, and the traffic seen each day via search engines is only a small part of the overall story, so as one would expect, it’s difficult to see an effect.

Fortunately, we can drill down into the data a little further.  Below are the findings when looking at Firefox download numbers solely through Google.  The numbers represent all downloads through that single channel, i.e., via both organic search results and paid advertisements.

Are these results surprising?

Sort of.

Our previous experiment (with a more limited scale), showed us two things:

  • The vast majority of people – who would normally click on our advertisements – will either click on our organic search result or find Firefox via some other method when we turn off our ads
  • Having our search campaigns turned on increased the overall pie of daily downloads by approximately 1%

This latest experiment suggests that that initial finding could hold true, but it also makes clear the fact that the answer to our original question (“do search campaigns aid Firefox adoption in any way?”) remains somewhat inconclusive.

We want to make sure we’ve done a fully rigorous job here, so please comment with your thoughts if you have a different take on these numbers and analysis.

5 responses

  1. Web Strategy Guru wrote on :

    These figures are surprising, I would have expected a more significant difference. Can you post some of the ads here? Maybe the communication needs tweaking.

  2. Harry wrote on :

    Firstly, thanks to Ken, Alex and Blake for producing such an excellent blog!

    The results of this study area really interesting. I’d love to see this study run again, over a longer period, and perhaps with a bit more detail regarding the actual ads and search terms.

    Also, alternating the experimental condition according to fixed days of the week is a bit suspect. You probably need to counterbalance day of week since this is probably going to vary in a weekly saw-tooth pattern anyway.

    I wonder if the last week of July accounts for a seasonal variation in a large country. E.g. students back to university, or a national holiday somewhere.

  3. Arya wrote on :

    I would say the results are expected. Think of this: You are not happy with your browser & want a better one. You either know that FF exists (power user), ask that tech savvy colleague/ friend or read some reviews (normal user) or call up tech support (novice). This 1% success rate is the users you are catching at a time when they are looking for a new (replacement) browser

  4. Abhi wrote on :

    Yes, things do make sense from the numbers. Most of the ad-generated clicks are made by those who are not really bothered to go to the site and download FF3 despite the buzz. If these users, amid the buzz generated are presented with an easy link to download the browser will take a shot and download the browser, 90% of the times either because they want to try out something new or something better. This ad driven marketing is almost like a last finishing touch or garnish over the already present marketing campaign in place. Hence the numbers!

  5. Aman wrote on :

    Hi, We at a large portal wanted to determine exactly what you are. Whether ad campaigns cannibalize or uplifts organic traffic.

    – Daily turning off turning on is imho not the best strategy as it doesn’t account for seasonality, news coverage, press releases, media presence etc.
    – looking at the data at such a global scale may not tell the story, Id look at campaign level or keyword level to see.
    – campaigns need to help uplift conversion for keyphrases where the natural rankings are not high on the SERP
    -Any website traffic will tell you that weekly data has a certain pattern and attributing downloads to just days of the week is oversimplifying