The Winning Snippet Is ….

In partnership with the snippet team, we’ve finished the first phase of snippet testing. (See: What’s the “Snippet”?) We’re running the most extensive snippet optimization tests ever conducted – we are learning a lot about the snippet and its audience.

There are a number of challenges. For one thing, impression and click data is not available in real-time. I get that data 12 – 24 hours after a snippet goes live. Then, analysis takes time. With December quickly rolling toward a close, a 48-hour testing cycle eats up precious time. Next year we’ll hopefully start to conduct testing much earlier.

Starting November 23rd we tested 16 variations of a fundraising snippet. The variants tested included both icon (static vs. animated) and text length (short vs. long). We also tested four different themes for the copy: 1) Public Good Over Profit; 2) Executive Pitch; 3) Factoid; and 4) Just $3.

We collected data on “click rate” (impressions divided by clicks) and “donor conversion” (clicks divided by total donors) to measure performance of each snippet. Each variation received at least 10 million impressions during testing.

Below is a table showing the 16 snippets ranked in order of best to worst performing. Even though 51% of the readers of this blog voted for Public Good Over Profit as the likely winner, the best-performing snippet theme by a large margin was Just $3. While the click rate was much higher for the winning theme’s animated coin icon (0.17% vs. 0.10%), the conversion to donor was statistically the same whether the icon was animated or static.

Stuff we learned (or that we confirmed):
  • Animated icons get significantly more clicks, but a high click rate doesn’t necessarily lead to more donations.
  • For donor conversion, longer copy beat shorter among the Just $3 -themed variations. It may be that because snippet donors are less familiar with Mozilla, they are more likely to donate if we provide more information.
  • Though privacy & security is the #1 issue for existing Mozilla supporters, the Firefox browser snippet audience doesn’t seem as inspired by the issue – at least in the context of a donation ask. Although, we know from the stopwatching.us snippet success that this audience is responsive to other calls to action about privacy & security.

These are just a few of the discoveries from the first phase of testing. We are already putting the findings into action for optimizing snippet donations. These test results also sparked a bunch of new ideas we want to test, and lots of new questions. Stay tuned.

Snippet Icon Snippet Text Click Rate Donor Conversion
 
Mozilla, the maker of Firefox, is a non-profit and we rely on donations and grants. If everyone reading this gave $3, we would only have to fundraise 1 day a year. Donate today.

0.10%

7.49%

 3Snippet_40x50px
Mozilla, the maker of Firefox, is a non-profit and we rely on donations and grants. If everyone reading this gave $3, we would only have to fundraise 1 day a year. Donate today.

0.17%

7.29%

Mozilla, the maker of Firefox, relies on donations. If everyone gave only $3, we could make the Web even better. Donate today.

0.08%

4.87%

 3Snippet_40x50px
Mozilla, the maker of Firefox, relies on donations. If everyone gave only $3, we could make the Web even better. Donate today.

0.20%

4.37%

 
Did you know that Mozilla is proudly non-profit and a leader in online privacy? Donate today.

0.05%

1.27%

 
Did you know Mozilla is a leader in online privacy? We build tools like Firefox to give you control over your data and to protect your rights. Support our work with a donation today.

0.12%

0.70%

 
We are a movement, not a business. Donate today to support our non-profit work to protect the Web and create value for everyone.

0.09%

0.69%

 lockSnippet_40x50px
Did you know Mozilla is a leader in online privacy? We build tools like Firefox to give you control over your data and to protect your rights. Support our work with a donation today.

0.24%

0.59%

 lockSnippet_40x50px
Mozilla, the maker of Firefox, is a non-profit and we rely on donations and grants. If everyone reading this gave $3, we would only have to fundraise 1 day a year. Donate today.

0.24%

0.49%

 
We are a movement, not a business. Donate today to support our non-profit work to protect the Web and create value for everyone.

0.21%

0.48%

 
We are a movement, not a business. Donate today to support our non-profit work to protect the Web and create value for everyone.

0.13%

0.46%

 
Mozilla is proudly non-profit. Donate today to support our work to protect the Web and create value for everyone.

0.09%

0.43%

 
“I believe in Mozilla’s mission to keep the Internet alive and accessible. We know an open Web is worth fighting for. You can help. Donate today.” – Mark Surman, Mozilla Foundation

0.13%

0.42%

 
“I believe in Mozilla’s mission to keep the Internet alive and accessible. We know an open Web is worth fighting for. You can help. Donate today.” – Mark Surman, Mozilla Foundation

0.19%

0.40%

 
Mozilla is proudly non-profit. Donate today to support our work to protect the Web and create value for everyone.

0.21%

0.36%

 
Donate to Mozilla today. “We are proudly non-profit and we protect the open Web.” – Mitchell Baker, Chair.

0.25%

0.35%