Bitcoin Donations to Mozilla: 17 Days In

Mozilla

33 responses

Just over two weeks ago Mozilla began accepting bitcoin donations. In the first three days our bitcoin donation form was live, we raised $1,600 USD in bitcoin, and to date we’ve raised about $5,000 USD. Here is the trendline:

bitcoin_dec6_update

We received numerous requests from donors that they wanted bitcoin featured right on our primary donation form. Given the volume of page views to that form (millions during the life of the campaign), I was concerned that adding any unnecessary text would distract donors and depress non-bitcoin conversions, the source of more than 99% of all our campaign revenue. So we decided to add “Donate with Bitcoin” text, and test whether it depressed conversion or not. Here is what that looked like:

We tested adding "Donate with Bitcoin" text to the form.

The test showed that revenue per visitor drops by about $0.07 USD. Here is the Optimizely graph showing a 7.5% reduction in revenue per visitor:

Testing Results: Bitcoin

Seven cents doesn’t sound like much. However, at scale, it adds up. Our donation form will get roughly two million more visitors before the campaign concludes on December 31st — which means adding “Donate with Bitcoin” would reduce income by about $140,000 — a significant amount.

At this time, bitcoin donations are not high enough to offset that lost revenue. We want to make sure bitcoin donors can find a link to give bitcoin, but this test suggests our primary donation form isn’t the optimum location.

33 responses

  1. Benjamin Kerensa wrote on :

    I just donated $6 USD worth of BTC :)

    1. Andrea Wood wrote on :

      Awesome!

  2. Tim wrote on :

    So why exactly do you think it would “depress non-bitcoin conversions”? It’s not entirely clear to me why adding the link would be a cause and not a simple correlation to the lower donation average.

    1. Andrea Wood wrote on :

      We are running an A/B split test between our control (our standard form without “Donate with Bitcoin”) and a form that includes “Donate with Bitcoin”. Nothing else is different, and the forms are exposed to the same audience at the same time for the same duration. Statistically speaking, “Donate with Bitcoin” would be the only reason for a change in conversion and/or revenue, once the test reaches statistical significance. We’re using a tool called Optimizely to conduct optimization testing.

  3. Erik Vold wrote on :

    It looks like this sample time is extremely short. That would indicate that there maybe a bias in the sample immediately. Looking at the graph there appears to be a huge wave of people on Friday, and I remember getting email blasts last week for donations…

    Perhaps the sample of people in your test are those that are more likely to respond to email blasts, and it’s just those who are affected by seeing a link for bitcoin?

    Also I would guess that a bitcoin image link on step 2 is possibly more user friendly than a bitcoin text link on step 1, and might have a less harmful affect on this group of people..

    And just to be clear, since I don’t see it explicitly stated, the revenue collected for the “Add Donate with Bitcoin” group does include the bitcoin revenue right?

    Finally I’m curious why there is a spike of bitcoin donations on nov 21st?

    1. Andrea Wood wrote on :

      Thanks for these thoughts —
      — Since November 1: Total revenue from donation form is $853,710; total revenue from all bitcoin donations is: $5,130
      — Our email audience is very small compared to the audience from about:home. We are continuing this experiment to get an even larger sample size, so we’ll post an update. It’s also likely that we have bitcoin users on our email list, so i’m not convinced there is a significant negative impact on our results because of exposure to an emailed audience.
      — Adding the “donate with bitcoin” prompt on step 2 is really interesting! We may add this to our test queue.
      — The “add donate with bitcoin” group does not include income from bitcoin right now, but because it’s only a few hundred $USD during the duration of the test so far, statistically speaking, it’s insignificant. Though it’s collected separately, we are incorporating those data into our analysis.
      — The spike in bitcoin donations on Nov 21st is due to the publicity from launching bitcoin on Nov 20.

  4. Voogru wrote on :

    This is a really short sample time, I’m willing to bet the result would have been the same with adding any text there, or perhaps the logos of various credit cards that are accepted.

    In fact that may be an idea, place all of the credit card logos there, and then the bitcoin logo is a link to donate with bitcoin.

    1. Andrea Wood wrote on :

      Thanks for the questions — the duration of the testing is irrelevant, reaching a statistically significant sample size is what’s important. Given the volume of traffic coming from our about:home page we can reach statistical significance relatively quickly. We are continuing this test for a little longer in order to gather more data, so expect an update in the next few days. Adding the logo to test is an interesting idea, we may consider it in the future. However, if the simple text link has an impact, a button will have an even larger impact on results, so we’re going to stay cautious and rely on data to guide us.

  5. travis wrote on :

    I started to donate $10 to Mozilla as i use it daily.. but I couldn’t find the “Donate with Bitcoin” link when I clicked the “donate” button in the top right corner..

    1. Andrea Wood wrote on :

      Sorry you’re having trouble Travis, please visit this direct link to donate bitcoin: mzl.la/bitcoin

  6. Voogru wrote on :

    “However, if the simple text link has an impact, a button will have an even larger impact on results”

    in that case, compare a version with just credit card logos against no credit card logos (original page), do not have anything related to bitcoin on that test page, and see how that impacts things.

    It will be interesting to see if revenue goes down with the addition of just the credit card logos.

    After that test, add the credit card style bitcoin logo to the lineup and then test that against just the credit cards.

  7. Geoffrey MacDougall wrote on :

    Cross-posted from Reddit:

    This was a standard A/B test. The platform we used was Optimizely.

    Wikimedia flagged the same challenge re: how more choice results in lower conversion: http://www.bitcoinx.com/wikipedia-founder-jimmy-wales-now-playing-bitcoin-receives-5-btc-donations/

    What this means: We haven’t found a way to integrate bitcoin into our funnel that doesn’t have a net negative effect.

    What this doesn’t mean: We’ve stopped experimenting with how to integrate bitcoin so it has a net positive effect.

    We engage in dozens and dozens of A/B tests during our campaigns. We’re fortunate to have a level of traffic where we can get statistically significant results in less than a day. We do all of this work in the open, blogging about the results (and our other activities) at fundraising.mozilla.org.

    We intend to keep playing with how to do this properly. Whether it’s new options on the page, separate forms, separate campaigns, etc.

    So ideas on how to make bitcoin have a net positive effect on fundraising are completely welcome. =) Best way to contribute to that conversation is through https://bugzilla.mozilla.org/.

    In the meantime, we’re still accepting bitcoin here: http://mzl.la/bitcoin

    Thanks!

  8. Eric Martindale wrote on :

    The Bitcoin payment shouldn’t be part of a different “flow”, it should be directly in step 2, “Payment”. The conversion rate will stay the same in your tests, and I hypothesize will result in a net increase of conversions overall. Get them over the decision to donate, and make selecting “Bitcoin” a part of the decision of “how to pay”. Treat Bitcoin equally, with other payment options.

    You should also consider publishing a dedicated Bitcoin donation address, and using that method to provide increased transparency on how the funds are used.

  9. Erik Vold wrote on :

    Ya an update would be great, with more time the test will be more accurate.

  10. Gabi wrote on :

    Wait, if I read your graph corectlly over the same period of time you had almost double number of (revenue) visitors. Do you have any idea what caused that spike? Isn’t the answer to this question also the main reason why your revenue per visitor decreased and not adding a small text link under the form?

    1. Andrea Wood wrote on :

      The testing period and the period including the revenue spike did not overlap. Also, all of the bitcoin donations until the A/B test launched were direct-referrals, meaning the links in articles were to our bitcoin donation page, not our general donation form. The A/B testing described in my blog post is for our general donation form.

  11. Robert Smith wrote on :

    Bitcoin is both an environmental disaster (thousands of megawatts of electricity wasted in a pointless arms race) and one of the best tools for child pornography ever created. I personally think less of any organization that affiliates itself with Bitcoin and I suspect many others feel the same. I urge Mozilla to cut all ties to this toxic “currency”.

  12. Spencer Ruport wrote on :

    “Bitcoin is both an environmental disaster (thousands of megawatts of electricity wasted in a pointless arms race) and one of the best tools for child pornography ever created.” – Robert Smith

    Actually Robert I believe you’re incorrect. If we’re going to be talking about the best tool for child pornography ever created the answer would undeniably be “the internet”.

    So if you’re taking a stance against a technology simply because it *can* be used for something disgusting and completely immoral I believe you are compelled by your own logic to cancel your internet subscription immediately.

  13. Robert Smith wrote on :

    No, child pornography is an unfortunate side effect of the Internet that decent people are working hard to stop. Greedy libertarian manchildren (aka Bitcoiners) happily overlook the fact that the product they shill for so desperately HELPS the producers and consumers of child pornography avoid detection. Indeed, illegal black market sales are the backbone of Bitcoin. Supporters harass above board businesses to accept Bitcoin payment processors to try to legitimize it and pump up the value, but nearly all of the volume is still on the illegal side.

  14. aw wrote on :

    Um, have you considered the fact that you got TONNES of pageviews simply because you offered bitcoin donations and it was posted on bitcoin forums?

    This is where the extra visitors came from, who either weren’t going to donate anyway, or were going to donate bitcoin. This would obviously depress the “average” non-bitcoin donation amount per page view.

    1. Andrea Wood wrote on :

      We can actually see traffic referrals in Google Analytics. We did see a spike in hits from bitcoin forums and news sites around our bitcoin announcement on Nov 20. From Nov 19 – Nov24 about 18% of all traffic was from a few forums/bitcoin sites and reddit (reddit made up about 16% alone). It’s since dropped significantly to about 1% of referral traffic. All of that traffic linked directly to our bitcoin form itself, not to our main donation form, so the traffic was not a part of our A/B tests, which were also launched more recently.

  15. Spencer Ruport wrote on :

    ” Greedy libertarian manchildren (aka Bitcoiners) happily overlook the fact that the product they shill for so desperately HELPS the producers and consumers of child pornography avoid detection.” – Robert Smith

    And hypocritical commenters on the Mozilla blog (aka Robert Smith) overlook the fact that the internet he relies on so desperately ALSO HELPS the producers and consumers of child pornography avoid detection.

  16. Amy wrote on :

    Robert Smith, by your logic, Mozilla also helps enable child pornography producers.

    I’m pretty sure child pornography existed prior bitcoin, also, I’m pretty sure the primary method that it was paid for was good ole US dollars.

    Either cash in the mail, or prepaid debit cards. Hell, even Amazon gift codes could be used.

    AMAZON = Enables child pornography!

  17. Robert Smith wrote on :

    You’re desperate to absolve yourselves of guilt by association by claiming that everything is basically the same, but there’s a big difference. Anyway, let’s run with your terrible analogy:

    The Internet = the highway system. Some small fraction of people driving on the highway are horrible child abductors with little kids in their trunks.

    Normal online payment = regular cars.

    Bitcoin = radar proofed stealth vans with soundproof trunks and automatically changing license plates.

    Now, you can argue all day long that your rape vans are just normal cars that happen to have lots of rape-friendly features and anyone who complains yet doesn’t want to tear up the highways is a hypocrite, but non-stupid people reject new technologies whose main advantages are to enable vile things.

  18. Darren wrote on :

    I visited the sight 3 times and didn’t donate because I didn’t see the bitcoin option.

    1. Andrea Wood wrote on :

      Hi Darren, we are still currently running the test to get a larger sample. You were likely routed by Optimizely (our testing software) into the “control” user experience to the form that doesn’t include “Donate with Bitcoin”. You can donate here: mzl.la/bitcoin (and we hope you do).

  19. Spencer Ruport wrote on :

    “You’re desperate to absolve yourselves of guilt by association by claiming that everything is basically the same, but there’s a big difference.”

    … You know guilty by association is a *fallacy* right?

    http://en.wikipedia.org/wiki/Association_fallacy

  20. Nick Sena wrote on :

    Correlation does not equal causation.
    Something i expect people in tech business to be well aware of.
    Also the time frame makes the whole thing statistically insignificant.

  21. David wrote on :

    Have you considered the fact that many people may have visited the form just to see the “Bitcoin Option”. These visitors may have had no intention to donate.

    If they got to the page with no bitcoin option they surely would have thought “wtf, I thought they were doing Bitcoin”, and refresh the page a few times until they saw the Bitcoin option page.

    Whereas if they landed on the form with Bitcoin option immediately they would have no need to refresh and visit the non-bitcoin page (which they did not know existed).

    This would heavily skew the results, especially if it was infact pre-announced that Bitcoin was being tested.

  22. Amy wrote on :

    “Bitcoin = radar proofed stealth vans with soundproof trunks and automatically changing license plates.”

    And what about preloaded gift cards? They can be bought with cash, and the numbers sent over the internet.

    Money is pretty fungible, you know.

    VISA/Mastercard/Amex gift cards = radar proofed stealth vans with soundproof trunks and automatically changing license plates

  23. garb wrote on :

    Has the 7c reduction been factored in to the c/c and or btc mining charges that would have been incurred with payment method? (ie c/c charge is relatively huge compared to btc, so is it possible people are donating less because they dont factor in/add more to compensate for charges?) and-or are people maybe donating less because the btc mining charge is taken from the donation amount, where as with c/c its added to the total – eg person wants to donate 5, when using c/c theyre asked to pay total of 5.97 so they add to their total. if using btc, only 4.93 gets through to you because the fee is deducted from the total during the process (and since its only small amount maybe people dont think to add more to compensate) …?

    i havent donated yet so dont know the process, yet. will be waiting for btc option (unless theres an id required as i heard. bit of an unnecessary and offputting extra step if true…)

    also, did you check donation amounts for same period last year? since its near shopping season maybe people are donating less anyway? is this the only drop in donation revenue youve seen in any time period?
    seems like a very short test period to make any serious conclusions…

  24. Meni Rosenfeld wrote on :

    (Sorry if this is a double post, the submission form was acting up)

    Apparently some commenters don’t understand how A/B testing, and controlled experiments in general, work.

    Mozilla didn’t compare a period in the past in which there is no button, with the current period with a button.

    Rather, during a specific period of time, Mozilla randomly chosen for every visitor whether to display a button or not. Then they measured the amount donated. Their results indicated a negative correlation between “there is a Bitcoin button” and the amount donated.

    Correlation between A and B can only be explained by:
    1. A causes B
    2. B causes A
    3. There is a common cause for A and B
    4. An artifact of small sample size.

    In our case:
    2. The donation amount cannot cause the button to not appear, since displaying the button was chosen randomly.
    3. There can be no common cause for the button and lack of donation, because displaying the button was chosen randomly.
    4. I expect Mozilla had enough visitors to make the results statistically significantly.

    Hence the only explanation is that the Bitcoin button reduces the average amount donated, at least in the period tested.

    Specifically regarding David’s comment, “window shoppers” are not paying regardless of the button, and thus have no impact at all on whether there is a negative correlation or not (they could affect the size of it however).

    All that said, I hope Mozilla will find a solution that will allow displaying the Bitcoin donation link more prominently, to allow people this very important method of donation.

  25. garb wrote on :

    Meni Rosenfeld, thanks for the extra info. Im of the opinion that your point 4 (“4. An artifact of small sample size…. 4. I expect Mozilla had enough visitors to make the results statistically significantly.”) is where the problem lies because simply displaying the btc option and attributing that to the cause of reduced donation amounts is similar to saying the same problem arises for Visa payers when they see an AmEx or Paypal option / logo.

    This seems counter-intuitive – which of course doesnt mean its wrong but that the simpler reason (insufficient sample size) is the more likely explanation.

    If the btc logo / donate option did somehow deter people from donating more, then it would speak to a lack of confidence trust in btc from mozilla donators – which again seems unlikely given that the more tech savvy are the more likely to be in favor of btc and are the type to donate too.

    Just my thoughts. Looking forward to see what mz does next. Maybe separating the payment methods onto separate pages where c/c payers are directed differently and never see the btc logo (ie btc could be displayed on a subpage of payment process if the user chooses ‘other payment option’ for eg)?