What did our advertising experiment really tell us?

I did an admittedly poor job of framing my last post in its broader context. We set out to discover the true effect of paid search advertising on a change (or increase) in number of Firefox downloads, and in the process, both the experiment and analysis were focused narrowly on one channel (US search) through which Firefox downloaders arrive (to the exclusion of all other channels).

In the broadest universe of Firefox downloads, what did our experiment tell us?

  • Over any two-day period, Mozilla sees about 1 million downloads of Firefox
  • Approximately half of these downloads are for our en-US version and about half are for our localized language versions
  • We focus our search marketing efforts primarily on the en-US market
  • Within this specific market, about 38% of downloaders come through the search channel (organic + paid)
  • Within this narrowly defined download channel – en-US search – about 75% is attributable to organic search and 25% to paid search.
  • Over the two-day periods in our experiment, we saw an average of 21K additional downloads as a result of our paid search advertising
  • Thus, by participating in paid search advertising, we are currently able to increase the total number of global Firefox downloads by approximately 2% (increasing 1 million by 21K).

A few last notes about this 2% estimate:

  1. It’s based on experiment numbers representing 80% of our en-US search traffic, so the true effect is closer to 2.5% (if we extrapolate the 21K increase) before considering the next point.
  2. When we have our paid search advertising “turned off,” the “loss” of downloads might be far less than the 21K indicated above. These prospective Firefox downloaders may simply be diverted to the “Other” traffic category (e.g., our affiliates program); unfortunately, we don’t have a good way to detect and measure such an occurrence. So, the true effect, rather than 2.5%, might be closer to 1%.

Summing it all up… we need to do more experiments, and from a cost-benefit standpoint, we need to decide if our marketing spend on paid search advertising is worth the additional 1% of Firefox downloads (alternately, we can work towards more effective strategies and tactics within this channel).

1 response

  1. Pingback from Another Mozilla Advertising Experiment - Bidding on Keywords < Blog of Metrics on :

    […] previously looked at an experiment with our online marketing programs here, here, and here. The results revolved around the interaction of regular (organic) search with paid search […]