In Part 1, we learned a little about our contributors and their test environments. As I continue my series of posts covering the results of the QA community survey, I wanted to shift the focus to volunteer participation in Part 2.
The following questions revealed a bit more about what people are most interested in doing and how they currently contribute to the Mozilla project. By understanding how current contributors are helping out with the Mozilla project, we can get a sense of what types of people fall into certain roles or responsibilities. That is extremely valuable as we continue to grow as a community and bring in new people that may not always know where to go or how they fit in. It’s up to us to guide them.
Let’s take a look at some of the questions and data…
How many hours per week do you spend on Mozilla QA related activities?
I currently spend…
I wish I could spend…
Those two bar graphs tell me a lot about the dedication and commitment our community has for the Mozilla project. It’s quite amazing that most of our QA volunteers spend at least 1-3 hours a week helping out. But what’s even more amazing is that a lot of them would be willing to spend anywhere from 4-10+ hours if they had that time! That is why the Mozilla project has been so successful and why the quality of Firefox continues to improve daily.
What Mozilla QA related activities are you working on right now?
There were many open text responses for this one, so I’ll just share a few that reflect the variety of activities people are involved in:
- Beta Testing and Nightly Testing
- I have a more developer oriented profile. I do some bug triage, and help
fixing by writing patches sometimes.
- Memory Performance, Bugs
- Confirming Bugs, Litmus test cases
- Try to reproduce what people are telling at bugzilla (Firefox only) and to
gather additional info about the bug, like regression date etc. I’m not a coder
and I only do things that are fun to do.
- Working on the FTR – For The Record; Mozilla’s media watch.
- Testday/Bugday support, QA Community Involvement, Seneca Club Moz,
- Looking into Norwegian translation
As you can see, our community is covering a broad set of QA activities… including general testing, bug triage, localization review, community leadership, and developer support. As we learn more about specific areas that some of our volunteers are interested in, we’ll be able to better spend our time organizing events and projects for them.
What Mozilla QA related activities are you most interested in?
The survey listed a few activities and gave volunteers three options for their level of interest:
- Maybe (with help)
- Not for me
The pie charts below show the distribution of interest for 4 major QA activities:
No surprise that “Testing” was the most popular, as it is the easiest way to contribute. And really, if you’re using Firefox daily and take the time to provide feedback or log bugs, you’re already testing it.
Quite a few people also showed interest in working with “Bugs”, but it’s clear that we need to do a better job educating them about Bugzilla and helping them with bug related tasks. Bugzilla is the center of the Mozilla universe, so the Mozilla QA team and community definitely should use it more often and take advantage of it’s many features to improve QA processes and product quality.
“Feedback Analysis” represents general Hendrix feedback and Talkback/Breakpad crash data analysis. The survey included both separately, but the results were very similar, so I put them into one bucket. We still have a majority of the volunteers interested in helping with this, but again the QA team needs to work with them to figure out interesting ways to look at the data and guide them as they comb through tons of user feedback and try to make sense of it all.
Then we have “Automation”, which wasn’t as popular as some of the other activities. Not too surprising, considering that working on test automation requires specialized skills and programming knowledge… and from the survey data, it looks like most of the volunteers that responded were just getting started with the Mozilla project and were mostly into general testing. That’s ok… we still have 33% that just need some advice and mentoring to get their hands dirty!Some write-in activities were “Documentation”, “Debugging”, and “Localization”… all of which are all very valuable contributions. Mozilla QA will definitely need help with all three when we start developing the new QMO site this Summer.
Which Mozilla QA mailing lists are you subscribed to?
One of the most successful Mozilla QA mailing lists we have had in a long time is the firstname.lastname@example.org list. Marcia and I created it last year to build a community of dedicated testers to help with alphas, betas, and pre-release builds. The list now has over 1600 members, so it’s no surprise that over half of our volunteers that responded are subscribed to it. Coming in second is our tried and true email@example.com list, which we have had for many years. I think both of those lists are going to be the QA team’s priority for keeping track of progress within the community and as major broadcast channels for big announcements and calls for help. But there are so any ways to stay connected, so it really doesn’t matter which list you’re on… we’ll reach you somehow.
As a member of the Mozilla community, what are the roles that best categorize your involvement with Mozilla? (Please select up to 3)
This question was intended to provide us with some idea of how contributors saw themselves within the community. One of the biggest challenges for the Mozilla QA team is to find the “right place” for some of the volunteers that come our way, so data like this definitely give us a good sense of where the popular spots are and hopefully with the feedback we have gathered, we’ll be better prepared to help new contributors find their way.
Some volunteers fall into multiple categories, as seen in the table below, which contains data from 93 responses. It’s interesting to see which buckets people put themselves in. Obviously most people see themselves as Firefox users… but it is great to have a healthy 34% of folks testing nightly builds as well. Another point worth noting is that 20% of the responses came from Web developers, one of the most valuable segments of the community when it comes to QA, as they are usually the people that are up to date on the latest fixes and their impact on various technologies and web sites.
Overall, it looks as if we have nice diverse group of folks making up the QA community. That definitely helps us get the most coverage, especially when we have nightly builds, Firefox 3 betas and security releases for 2.0.0.x all going on at the same time!
Part 3 coming soon…
I hope my first 2 posts have provided a good snapshot of what types of people make up the QA community and the various activities they participate in. The data we have gathered is meant to give us a base to work with as we try to find better ways to engage new contributors and improve our processes and tools to encourage more collaboration. Next time we will take what we have learned so far in Parts 1 and 2 and try to make sense of the direct feedback we have received through the survey. There are a lot of great ideas and requests, so it will be nice to share that with everyone. Part 3 will focus on bringing all of that together to create action items for the QA team to improve community efforts in 2008.