Our end of year fundraising campaign has finished now, but while it’s fresh in our minds we still want to write up and share the results of some of the A/B tests we ran during the campaign that might be useful for others.

At Mozilla we strive to ‘work open’, to make ourselves more accountable, and to encourage others to build on the things we have done. In the same way that open source software saves thousands of developers from writing the same code thousands of times over, we hope our ‘View Source Fundraising‘ can be built on and improved upon by many organizations facing the same challenges today.

This post is looking at logos on our donation form.

Should we use the Mozilla or Firefox logo on our donation form?

It’s possible that one or the other of these two logos resonates better, or has a stronger affinity for the kinds of users who choose to support our End of Year fundraising campaign. But we didn’t know which.

The majority of traffic to our fundraising campaign comes from Firefox users so we thought the Firefox logo might be more meaningful to them. Equally, people know Mozilla as the organization behind Firefox, so maybe that logo is best.

Luckily, we can test this rather than guess.

We tested these variations of the donation form

The results (part 1)

We compared the performance of each logo variation for conversions and revenue per visit, but this test didn’t produce the kind of clear winner in both categories which we usually look for.

However, removing the Mozilla logo looked to have promising (but not yet statistically significant) results in terms of revenue per visitor.

The test did have one clear loser: Using both the Mozilla and Firefox logo at the same time had a significant negative impact on conversion rate and revenue. We can only speculate about exactly why this is but we know for sure not to make that the default logo on our form.

Possible reasons the combined logo had such a negative impact include:

• Visual complexity puts people off (this is a recurring discovery in many A/B tests we run)
• Thinking about two brands simultaneously increases cognitive load, causing some abandonment
• Perhaps lack of clarity or confusion about who the donation is going to

Following up with a second test

In our logo test above, the final variant where we removed the logo showed some promise and we wanted to explore this idea further.

So we started a new test with fewer variables so we could get results quicker.

The Mozilla logo actually appears twice on the page. Once in the typical ‘branding spot’ in the top-left, and once in the top-right in our ‘Tabzilla’ feature (which is a logo with a drop-down menu connecting various Mozilla properties).

Results of this test

Removing either the logo or Tabzilla has a positive effect, but the strongest effect is seen when removing Tabzilla. A 3.2% increase in conversion rate and a 3.4% increase in dollar per visit.

Conclusion

• We should remove the Tabzilla feature (with duplicate Mozilla logo) from our donation pages (which we did)
• Along with the UX change which has a measurable impact on conversion as shown in this test, loading Tabzilla from an external source (it’s hosted on another domain) has a performance impact on page load time, so by removing this at source (rather than just visually as we have done in this test) we can make further gains in page load speed, which also has a positive impact on conversion rates.