Mozilla Online Advertising – an Experiment

Perhaps one of the business world’s easiest arenas for running experiments is online advertising (e.g., pay-per-click search advertising). With the click of a button, you can easily vary one of many variables – messaging, keywords, budget, etc. – to instantly understand its effect.

Over the past couple of months, I’ve been collaborating with David on some different ways to optimize our online advertising efforts. The goal includes improving the downloading path/experience of new Firefox users, as well as understanding and optimizing the investment Mozilla is making in this channel.

One upcoming experiment relates back to funnelcake. Some of you may remember that the source of downloaders looked this:

Thanks to the popularity of our download page and some recent search optimization efforts, the Firefox download page occupies the #1 organic search result in most major search engines (when searching for “Firefox”). With that in mind, one natural question that arises is: if we did zero pay-per-click (ppc) advertising, would our organic traffic (i.e., unpaid search) rise by a corresponding amount? In other words, given the following scenario:

  • I am prospective Fx user
  • I visit a major search engine searching for Firefox
  • I have a propensity to click on paid ads

If I am not presented with an ad in the search results (an official Mozilla ad directing me to www.mozilla.com/firefox), will I instead click on the first organic search result (which also directs me to www.mozilla.com/firefox)?

So, from a results perspective, if we turn off our ppc advertising, we’ll be looking to see if the pie chart above morphs into the chart below (note the 38% organic picks up the 10% ppc from above). While the answer will likely be a strong “no,” we wouldn’t be responsible marketers without at least giving this question some thought and understanding how this dynamic impacts our true cost of acquiring a Fx user through online advertising.

We look forward to soon sharing our methodology and findings.

10 responses

  1. Matthew wrote on :

    What piece of software did you use to make those pie graphs?

    ps. the comment box icons are really confusing

  2. John Smith wrote on :

    I think I should stop reading this blog. It seems to have bought into the whole “advertising is a good thing” idea.

  3. Andy Burns wrote on :

    When performing such an experiment, is it wise to announce publicly the findings you are hoping to see before performing the experiment? Does it not leave you open to deliberate attempts to skew the result?

  4. Pingback from Mozilla Online Advertising (Part 2) - Experiment Findings and Marginal Cost < Blog of Metrics on :

    […] our previous post, we discussed an upcoming experiment related to Mozilla’s online advertising efforts. We saw with […]

  5. Pingback from Mozilla Online Advertising (Part 2) - Experiment Findings and Marginal Cost · Get Latest Mozilla Firefox Browsers on :

    […] our previous post, we discussed an upcoming experiment related to Mozilla’s online advertising efforts. We saw with […]

  6. Pingback from Organic and Paid Search - How Opposites Interact | Giant Spatula > a Rolnitzky blog on :

    […] Ken and I set out to answer a question about our search program: how does organic search (meaning results in a search engine […]

  7. Pingback from Organic and Paid Search - How Opposites Interact · Get Latest Mozilla Firefox Browsers on :

    […] and Paid Search – How Opposites Interact Recently, Ken and I set out to answer a question about our search program: how does organic search (meaning results in a search engine […]

  8. timfry wrote on :

    Andy – I would assume that announcing it on his personal blog that is not linked from many places (I found it from the Freakonomics blog on the NYT) will not have a noticeable impact on the thousands/millions of individual decisions made over the course of this experiment. For those that will find this most will already be using Firefox or be set with their browser of choice. Shouldn’t be a big problem

  9. Pingback from Another Mozilla Advertising Experiment - Bidding on Keywords < Blog of Metrics on :

    […] previously looked at an experiment with our online marketing programs here, here, and here. The results revolved around the interaction of regular (organic) search with paid […]

  10. Pingback from Another Mozilla Advertising Experiment - Bidding on Keywords · Get Latest Mozilla Firefox Browsers on :

    […] previously looked at an experiment with our online marketing programs here, here, and here. The results revolved around the interaction of regular (organic) search with paid […]