Participation Lab Notes: Short Simple Tasks Increase Engagement

The Mozilla Participation Lab is an initiative that focuses on experimenting and surfacing new approaches to Participation. Read more about the Lab on our wiki page.

Every year, about 7 million people come to the contribute page on Mozilla.org looking for information about how to get involved with Mozilla. These visitors represent an exciting opportunity to increase the number of long-term relationships Mozilla has with people who are passionate about the open web.

Right now, however, only about 0.76% of those people ever register to contribute. Even once they’re registered, how we connect them to their contribution area of interest is far from optimized.

In partnership with the Mozilla.org team, the Participation Team has set out on a medium-term project to try and maximize the potential of this page. Our aim is to make it a better tool for potential contributors, Mozilla, and the mission.

This first experiment set out to figure out how to increase engagement with tasks presented to a visitor after they said they were interested in contributing. We designed and ran an A/B test over a few weeks, presenting visitors with different types of tasks.

tl;dr

  • we were able to increase the engagement rate to 60% (from 15%) in selecting tasks
  • visitors preferred simple tasks to challenging tasks, and shorter tasks to long tasks
  • presenting more options (6) versus fewer options (2) increased the number of people who chose to pursue a task

Keep reading to learn more about the results and the future iterations and experiments we have planned.

Experiment

In this A/B test we created four new versions of the contribute sign-up page. Each page presented viewers with a number of tasks categorized either as simple/challenging (variation a) or as taking a little time/more time (variation b). For each of these variations we displayed either 2 or 6 tasks per page.

Variation B : Simple vs. Challenging

Variation B : Simple vs. Challenging

Variation A : Little Time vs. More Time

Variation A : Little Time vs. More Time

For the purpose of this first test we did not track engagement past choosing the specific task, and we did not track engagement with individual tasks, measuring only the category of the tasks selected (eg. simple/challenging or little/more time).

Results

The major findings from this first round of testing were that visitors to the contribute page preferred simple tasks to challenging tasks, and shorter tasks to long tasks. We also found that presenting more options (6) increased the number of people who chose to pursue a task instead of selecting the “not ready” button. Finally across all of the page variations we found the percentage of people who engaged with the pages (either by choosing a task or selecting not-ready) was 60%, much higher than the 15% engagement rate of the original page.

Based on all of the results we can infer that majority of visitors to the contribute page prefer easier, low-barrier tasks, and prefer a handful of choices to just 2.

However there were still a minority of individuals who expressed a preference for more challenging tasks — they may represent skilled contributors who are ready to become deeply engaged in complex projects. In coming iterations of this experiment we will be exploring the level of readiness and skill of these contributors, and whether individuals from this group are more likely to become core contributors than those who selected simpler tasks.

Overall, the enormous engagement rate by both groups further supports the amazing opportunity latent in this page.

Next Steps

Over the coming months we will undertake several rapid iterations on this test. There is a great deal still to be learned, however we believe that through experimenting with participation in venues like this we can unlock huge potential to help people engage with Mozilla in high impact ways.

Over the next few weeks the Participation team will be working to design a second experiment and we will focus on:

  • Enhancing our understanding of the specific kinds of tasks contributors are most interested in
  • How the task completion rate varies among types of tasks and levels of difficulty and what this tells us about the contributors we are able to attract through this page
  • How wording and framing of tasks effects task selection and completion

Other topics we will seek to explore in the longer-term include:

  • Understanding best practices for how to setup second and third tasks to engage contributors in a more long-term way
  • How presenting localized tasks affects completion rate How to identify and present tasks with the maximum appeal to contribute page contributors
  • How to provide “off-shoots” for the non-typical contribute page contributors (such as those who want to contribute code)
  • How to increase the conversion rate from the homepage to the contribute sign-up page If, when and how to collect email sign-ups and what the process is for engaging contributors by email

This project is currently owned by Lucy from the Participation Team in conjunction with the Mozilla.org Team. If you have any questions, ideas for future tests, or if you just want to chat about the future of this page please do not hesitate to get in touch!