Distributed Qualitative Analysis for the Firefox Behavioral Segmentation Study

The Mozilla User Research team has begun a large-scale, behavioral segmentation of Firefox users. The goal of the project is to identify Firefox user types in order to understand their behaviors, motivations, and attitudes. While we are focused on users who identify Firefox as their primary browser, we are also interested in understanding how users use other browsers (on the desktop and on mobile) and how user access the internet in general. The results from this research will be used by the Firefox team to make decisions about the Firefox roadmap, its priorities, and new features.

We have started an initial version of the study in North America. The study is divided into three components:

  1. Qualitative in-home interviews with demographically-relevant Firefox users in three major markets,
  2. A week-long diary study with Firefox users to understand their daily habits,
  3. A quantitative study to validate and expand upon our findings in the first two components.

We have completed the first two components and are in the process of analysis.


What makes our analysis process interesting for this project is that our user research team is distributed geographically along the West Coast. The challenge in any analysis of qualitative data is to bring together coherent trends and derive insights. Qualitative analysis can be an intuitive process as much as a logical and analytic one and it is helpful for the team to be colocated in the same space, working together. In order to be successful working part of the time separately, it was necessary to have a clear plan coordinating our individual processes and goals before we started the actual analysis.

Here is how we tackled the process…

We broke up our participants based on the locale each of the main research leads visited. As a group, the team agreed on a clear set of categories of user data and themes we were looking for so that we could bring together similar data to work with. Next, each group reviewed the video and notes for each participant in their locale, coding answers and observations about behavior, attributes, and motivations based on our agreed-upon categories. Based on our notes, we built portraits of each participants’ behaviors, attitudes, and beliefs. On post-it notes, we wrote out attributes and observations about the participants and clustered similar items together. Finally, based on these clustered attributes, we came up with rough groupings of similar Firefox users in our locales.

Starting analysis in our Portland workspace.

A week later, the team convened in our Mountain View office for an intensive work week where we took over a room and made it into a hub of information for our analysis. Each team member presented their brief portrait of each of their in-person participants. Next, we discussed the rough groupings we were derived from our individual clustering exercises. For reference, we used an excellent meta-analysis study of similar user type groups generated by similar studies. Fortunately, each team had significant overlap in terms of our general groups, making the process for coming up with user types a bit easier.

Mountain View Analysis Space

 

Finally, we took important attributes, motivations, and behaviors from our participants and mapped them on a wall and on a spreadsheet. Working with our quantitative researcher, we were able to determine which attributes were statistically salient differentiators among the participants and also which attributes correlated statistically with other attributes.

Wall for comparing behaviors, attitudes, motivations among participants.

 

We are continuing to work through our analysis the rest of December and will post soon on our results.