Testing, testing and more testing

While the team working on our End of Year fundraising campaign are heads down iterating on form design, localization, email writing and much more I wanted to take a quick moment to report out on how the numbers are looking so far.

In short, it’s going really well.

And the number I’m most excited about today is 10%.

Screen Shot 2014-12-02 at 20.26.27We’ve shipped another new version of the donation form since the last blog post I wrote and this has increased conversion rate by a further 15% and fundraising income by 18%. We’ve also tested a set of headline variations and the winners of that test pushed the fundraising income up by another 16%. We will continue to run tests and to report back on this blog.

Screen Shot 2014-11-28 at 12.36.53Right now, our biggest audience is the percentage of Firefox users who are seeing our fundraising appeal on the about:home snippet, so this is where we’re focusing our testing efforts. If you haven’t seen this ask on the Firefox homepage yet, you hopefully will do soon.

Screen Shot 2014-12-02 at 22.12.41

The audience who are responding to the snippet ask and landing on our donation page are converting at over 10% right now, which I can tell you from many years testing and optimizing donation forms is really good. I’d usually be happy to see a 10% conversion rate for a donor responding to an emotional TV ad where the available space to explain what you’re doing and why you need donations is much greater than what we’re working with here.

The snippet is a unique piece of ‘web real estate’ but given the limited space for storytelling it should be more comparable with a ‘cold’ banner advertising ask, where most fundraisers would be happy seeing even a 1% conversion rate. To get from a two sentence fundraising ask to a 10% donation rate is pretty special.

If we can keep tuning the performance of this donation page, and keep iterating on our fundraising messages on the snippet this year, we’re quietly getting exciting about breaking quite a few Mozilla fundraising records. But we can’t rest at this point. The audience will change, and messaging fatigue will kick in, and we will have to battle these challenges with data and design. Also, if we rest now we might be missing out on opportunities to make this process work even better, and that’s not our style.

Screenshot_2014-11-26-11-31-29

Alongside the success we’re seeing for traffic coming from the snippet, we’re seeing some other promotional channels converting at a much lower rate. So we’ll be iterating on these user journeys in the coming weeks to see what we can do there. One particular example is mobile traffic, where our existing form is fully functional on a mobile form factor, but the conversion rate is much lower than we hoped. So we’re going to test some other approaches here and we’ll report back and what works best.

We’re also working hard to roll out the improvements we make from testing to the localized fundraising experiences as quickly as we can each time we conclude a test.

That’s it for today, but keep an eye on this blog for more updates.