Jay's Blog Thoughts on Mozilla marketing, err…engagement.

One of the main reasons for conducting our QA survey was to gather feedback from contributors to find out what they think works (in terms of events and activities) and what they want to see the Mozilla QA team do to grow, empower, and improve the community going forward.  Following Part 2, which explored participation, we now look at some direct input from the community to wrap up the series in Part 3.

The following 3 questions provided plenty of data and ideas for the QA team to consider as we move forward…

What are/were your biggest obstacles to working on the Mozilla project?

Anyone can download Firefox and use it or “test” it, but what comes next if you want to get involved with the Mozilla project?  For potential contributors, that’s not always an easy question to answer.  The data below represents responses from 93 volunteers…they were asked to select their top 2 obstacles.

Though the Mozilla project is very open and there are plenty of opportunities, it can be a daunting task to figure out where you fit in or how to get started… and the majority of those that responded agree:

With so many wiki pages, newsgroup posts, blogs, and forums to comb through to figure out what is going on, the QA community can easily get lost.  That is why I have been busy working on the QMO (quality.mozilla.org) redesign project.  We hope the new QMO (coming Summer 2008) will provide a launch pad for anyone that wants to get started with Mozilla QA.

The Mozilla QA team can also address two other big obstacles with a combination of a well organized QMO site and more collaboration with the community.  While the site itself will provide better documentation and tips on getting involved, the team  hopes to be more proactive in our communication with contributors.  Our goal is to build relationships that will provide the mentorship and guidance that many people need when they first join the community.  QA does not have the benefit of a module ownership system (like developers) and therefore it is up to the QA team to be more visible and step up to become leaders within the community.

There are also a few obstacles, like time and software/hardware needs, that we can’t always remedy… but we are trying to help as much as we can.  Many contributors have already benefitted from Mozilla’s Community Giving Program and the QA team is always looking for better ways to schedule our events and projects around our international community so that people in all time zones have a chance to participate.

With a better understanding of some of these stumbling blocks, we can better prioritize our efforts and continue to work hard to make it easier for people to get involved and help them become effective contributors to the Mozilla project.

How do you think the current Bug Days and Test Days can be improved?

Bug Days have been around since the early days of the Mozilla project when Asa helped organize weekly bug triage events, a tradition we continue every Tuesday.   Test Days are every Friday and encourage more community test coverage for critical bug fixes, imporant milestones and new releases.  We have had decent success with both events over the past few years, but know we can do better with input from the community.

There are some common themes in the feedback we received.  I have summarized a few of them below:

1. Different days, better time slots
2. More interaction with QA team
3. Focus on helping newbies
4. Plan specific Bugzilla activities, like the unconfirmed bug triage projects.
5. Better documentation and tutorials on some basic QA “lessons”
6. Improved feedback/reporting channels
7. Allow localization of content and tools
8. Make it easier and faster to participate and accomplish tasks/goals
9. Weekend long events, open scheduling, etc.
10. Innovate communication and collaboration with community

The Mozilla QA team is actively working on a few of them now, and we have definitely either thought about or have plans to address the rest.  Now that we have a basic idea of what the community wants to see, I plan to contact a few of the contributors to get more details and then work with the QA team to make them happen!  This is a great start to improving our collaboration with the community, and I can’t wait to see some of these improvements in 2008.

Imagine you are visiting QMO (quality.mozilla.org) for the first time. What would you expect to see?

The “alpha” experiment with the current QMO site has provided good lessons and inspired great ideas for the “official” QMO site, but I also wanted to get some realistic expectations directly from contributors to make sure the design direction I have taken with the visual refresh and site architecture will meet the needs of our diverse community.  Below are a few thoughts directly from those that participated in our survey:

•  Tests that can be run without using Litmus, a functional extension to interface with QMO
  • DONE!  See the Mozilla QA Companion
• Instructions on how to collaborate and perhaps FAQ section.
• Events, Documents about how QA works/getting involved, contact methods.
• Results from the work done. (counters of bugs squashed during a bug day etc) (Metrics or statistics)
• Link to the latest beta version and to the latest nightly version of each
  branch. Explanation of branches and version numbers for branches.
• A short description about the website, its objectives and role in the Mozilla
  community
• Have a download link for FF3.0 Beta. Do you actually read all these comments?
  • YES, and thank you for the idea, see the very top of QMO. We have been doing that since Beta 2.
• Very real as I haven’t been there yet. I would expect to find a nice UI (last priority), but lots of documentation on how to be a good QA and QA Community Member (E.G. what to do and how to respond to questions and what to look for) and updated lists for what is the top priority like do you want to get rid of a lot of bugs this month of confirming new bugs (E.g. the QA staff can confirm). Or do you want us to push Firefox and file as many non-dupe bugs, that kind of list. I would also want to find a portal or page that lists all of the Firefox IRC channels, how to get in touch with the devs and QA staff and especially a list of the sites like I didn’t know there was a test day site/blog would be nice
  for all of these sites to be well documented in one place for easy resource access and such.

I wanted to throw that last one in there because it’s comforting to know that everything I have gathered so far about the needs of the community is pretty close to what people that have never visited the site might expect to see.  Some of those other ideas will also help me refine the layout and prioritize the content we need to create for the site.  It is awesome to know that this feedback will have a direct impact on the success of the new QMO when we launch this Summer.

That’s all I have in terms of the feedback we have collected.  The QA team has a lot to think about and hopefully when the Firefox 3 release madness is over we will all be able to dedicate some time to addressing the issues and exploring the new ideas we have collected through this survey.

To wrap up this series, I wanted to share with everyone a few responses to the question below to remind us all why we do what we do as part of the Mozilla community…

Why do you dedicate your time to help with the Mozilla project?  What about Mozilla QA keeps you engaged in the community?

• I spend a lot of time using Firefox, as a user or Web developer. That’s why I’m interested in contributing back and helping make it even better.
• I want to support a product I use daily and love.
• I get some sort of sick pleasure out of it.
• I love mozilla* but I hate bugs in it
• It’s FLOSS, so I can.  The browser is a very important piece of software, probably the most widely used application.  The net/web is where it’s happening.
• I like to improve Firefox and as a future programmer, also learn about the software development process.
• I like firefox. It saves me from having to use IE 4/5/6/ and 7.
• Another browser needs to exist to put pressure on Microsoft, it drives innovation.
• To help Mozilla produce great quality products and the QA team did great job in hosting QA events.
• I was and I’m involved in many testing programs from software to hardware. I’d like to know new technologies and applications and I use this to improve my professional training and to be prepared for the next
• I’m a Firefox fan since Phoenix 0.3, got addicted some way.
• Because there are many people out there that need to be protected and updated and i love to help others achieve this goal
• So many people devoting time, working together (sometimes/always?) without pay. it just feels like its a project by the community for the community.
• Lots of people use Mozilla products, especially Firefox, and it’s very rewarding to be part of something so big and significant for a lot of people.
• Connected to my class at school, but also it’s a great community and a positive learning experience. since becoming more involved with mozilla i have learned a lot about real world development.
• Ive got nothing better to do

There’s a lot more where that came from, along with all the details from the survey results… so if you’re interested, check out the complete report online or download the pdf.  Some of the charts and tables aren’t very useful (limitation in the surveygizmo reporting), but all the data is there.

I hope this series of blog posts provided some interesting information about the Mozilla QA community and improved our visibility within the Mozilla world.  People sometimes forget that there are many QA contributors out there helping out that you won’t see in any Bugzilla comments or CVS checkin logs.  They are sometimes invisible to the rest of the community, but their contributions are just as significant to the Mozilla project and critical to the success of products like Thunderbird and Firefox.

My goal is to make sure the Mozilla QA team and community never feels invisible…

This entry was posted on Thursday, April 3rd, 2008 at 4:35 PM and is filed under Mozilla QA, QA Community. You can follow any comments to this entry through the RSS 2.0 feed. You can skip to the end and leave a comment. Pinging is currently not allowed.