{"id":538,"date":"2014-12-26T21:30:25","date_gmt":"2014-12-27T05:30:25","guid":{"rendered":"http:\/\/mozfundraising.wpengine.com\/?p=538"},"modified":"2019-02-27T07:41:28","modified_gmt":"2019-02-27T15:41:28","slug":"when-testing-meets-localization","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/foundation-archive\/mozilla-science\/when-testing-meets-localization\/","title":{"rendered":"When Testing Meets Localization"},"content":{"rendered":"

Andrea\u2019s recent post talked about the importance of this fundraising campaign being global, and I wanted to follow up on this with some notes about how our A\/B testing strategy, and the volunteers who work on our localization have overlapped.<\/p>\n

To start with, I want to note that a huge chunk of this fundraising appeal is only possible thanks to the Mozilla L10n (localization) community. Without their incredible support, we would be a long way short of our current fundraising total. These contributors are donating their time and energy to support Mozilla in a way that has real and measurable impact.<\/p>\n

Because they are giving up their free time to do things that no paid-contributor working on this fundraising campaign has the ability to do, we never want to rush or make demands about when these translations need to be done by. And we don\u2019t want to waste L10n\u2019s time asking for more translations than we can actually use.<\/p>\n

At the same time, we\u2019re working on a rapid-fire A\/B testing strategy for the design and content of our fundraising appeal as seen by English speaking audiences. The team are writing copy, animating images, and building systems which may only be live for a day before we replace them with something that performs better. Much of the work we do here will be discarded, but we know that\u2019s OK because together we\u2019re creating the combination of content and systems that has the most impact we can.<\/p>\n

These two approaches are at odds with each other:<\/strong> minimizing content creation tasks versus mass discarding of content. And we\u2019re trying our best to balance the needs of these two approaches this year.<\/p>\n

The way we\u2019ve worked is to test extensively against the EN (English) language versions of our Snippet appeal and donation form and whenever we make a significant breakthrough in conversion, roll those features out to the equivalent localized versions. We know this isn\u2019t the optimized version of localized optimization (that\u2019s a fun sentence!) but it\u2019s had real positive impact.<\/p>\n

Here is a graph of income from our French supporters:<\/p>\n

\"Screen<\/p>\n

The graph above shows what happens when we first localize our default snippet like this example based on our best performing EN text:<\/p>\n

\"Screen<\/p>\n

And the second bump on the graph is when we rolled out the snippet design which worked so well for the EN audience, like this example:<\/p>\n

\"Screen<\/p>\n

The thing is, we know that the best way to ask for donations when talking to an English speaking audience isn\u2019t the same when speaking to a German audience. It\u2019s not even the same across English audiences. As an English English speaker, I\u2019ve been slowly acclimatized to all the Zs that my American English speaking colleagues like to use (in words like localisation). But there are bigger differences we barely have the time to think through in the thick of the campaign, let alone build and test. For example, in the UK, most fundraising revolves around regular giving (recurring monthly gifts) but that\u2019s not the case in the US. And this End of Year campaign is connected to the major giving days in the US which connect to the tax year and drive those cultural norms. That\u2019s not the same in the UK, though we have a lot of fundraising appeals that link up to Christmas.<\/p>\n

For every test we ran this year for our \u2018Generic English\u2019 audience, the \u2018optimum\u2019 solution for local audiences is different. Red buttons work for some, not for others. \u2018Donate Now\u2019 was a good headline in English, but it comes across as shouting in Indonesian.<\/p>\n

To truly optimize for every audience, the number of testing variants quickly escalates beyond something we could manage with a team our size.<\/strong> And so everything we do in a campaign like this is a judgement call about where we can have impact given our resources. This is true of all testing strategies but localization and testing have a multiplying effect on the number of variants of any given piece of content.<\/p>\n

While we\u2019re not at \u201clocalized optimization\u201d yet we have had real impact on our global fundraising this year, mostly due to the L10n community, with a little multiplying bump from some of our testing results.<\/p>\n

Maybe one day we can find a way to run more localized testing experiments. I\u2019d love to bring the L10n communities more directly in the campaign process and enable them to run their own design and content tests. I think that would be the most exciting and Mozilla-like way to scale this process. It has lots of challenges and could get really confusing. But I think it’s exciting.<\/p>\n

But until then, I think we\u2019re striking a nice balance of testing and localization. If you’re working on a similar fundraising appeal, I think this is a valid approach to take.<\/p>\n","protected":false},"excerpt":{"rendered":"

Andrea\u2019s recent post talked about the importance of this fundraising campaign being global, and I wanted to follow up on this with some notes about how our A\/B testing strategy, and the volunteers who work on our localization have overlapped. … Continue reading<\/a><\/p>\n","protected":false},"author":144,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[327515],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/posts\/538"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/users\/144"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/comments?post=538"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/posts\/538\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/media?parent=538"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/categories?post=538"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/tags?post=538"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}