Here’s a quick quiz to start our discussion:

If you acquire three customers through online advertising and your total cost is $3 and your reporting system shows your cost per acquisition as $1, what was your true cost to acquire each customer?

Hint: the correct answer is not $1 to acquire customer A, $1 for customer B, and $1 for customer C. You’ll find our answer at the end of this post.

In our previous post, we discussed an upcoming experiment related to Mozilla’s online advertising efforts. We saw with funnelcake that approximately 38% of Firefox downloaders originate from search engines, so understanding users’ behavior within this channel – along with the full costs and benefits of our paid search advertising – is critical from a marketing perspective.

The question we set out to answer is this: if we turn off our pay-per-click ads, will a user who would have normally been expected to click on our paid ad simply click on our regular (i.e., organic) search listing?

Our problem solving methodology somewhat mirrored the flicking on/off of a light switch, as we conducted the experiment in such a way that we were able to largely isolate the effect of having our search ads on/off on a change in the number of Firefox downloads. Here’s the methodology we used:

• Utilized a Monday through Thursday time period within a particular week. We’ve previously noted that internet traffic appears fairly consistent across these days.
• On alternate days, we turned off our paid ads. For example, one week our ads were active on Tuesday and Thursday and turned off on Monday and Wednesday. In the other week, we reversed the on/off days. We alternated like this to reduce the chance of any biasness or exogenous factors affecting the experiment.
• The audience involved includes only users downloading the en-US version of Fx
• The results from Week 1 and Week 2 are presented below. The pie charts answer the question: Where are downloaders coming from?

Here’s what we found in Week 1:

Here’s what we found in Week 2:

How should we interpret these pie charts?

Approach A – How much of that green pie slice are we still able to capture with zero ads?

• You’ll see in Week 1 that 19.3% of downloaders came through organic search and 7.4% arrived via paid search. When we turned off our paid ads, our organic search did not pick up that entire group of 7.4%, but it did pick up most of that group (24.6-19.3, or 5.3%).
• Taking 5.3/7.4 tells us that we capture 72% of would be paid ad downloaders through our regular (i.e., organic) search listing when we turn off our paid search ads.

Approach B – Total effect on aggregate numbers

• What was the effect on total number of downloads (just the raw number)?
• In Week 1 we saw 142K downloads through organic search when our paid ads were turned off. And when our paid ads were active? We saw 114K downloads via organic search and 44K downloads through the paid search ads.
• So, we captured a total 158K downloads vs. 142K by having our paid search ads active, or an 11% increase when considering just total numbers.

What does this mean for Mozilla marketing and our future efforts related to online advertising? While the findings are both surprising and telling, it’s important to remember that the steps outlined above were only an experiment. We’re not going to dramatically transform our business practices overnight without fully considering all ramifications (i.e., we’re not going to instantly turn off all Firefox advertising tomorrow). That said, these findings do tell us that we need to seriously rethink our existing online advertising strategies and tactics.

David and I look forward to returning to this subject again in the future, and we encourage your thoughts and feedback.

Lastly, returning to our original quiz, Mozilla’s cost of acquiring three new users would look like this (please note: the dollar figures are hypothetical):

customer A = $0, customer B = $0, and customer C = $3

How can we explain this? Marketers have been taught to think of costs associated with acquisition marketing as average costs (e.g, cost per acquisition). Borrowing a term from economics, we would benefit by transforming our thinking to that of marginal costs. In other words, marketers should be asking themselves, “is it worth $3 to acquire customer C?”, not “is it worth $1 to acquire each customer?”.