In Part I we discussed a framework around understanding the Firefox user experience at the moment a person installs Firefox. In Part II we walked though our recent initiative – soliciting feedback from users when canceling out of the installer – and provided some high-level results. With Part III we now want to dive into the details of the written feedback provided by users and discuss how these insights will translate into decisions, actions and next steps.
As we highlighted last time, about 330 users left feedback. The vast majority of people also provided a written comment (which was not required). In looking at the dashboard, there are some interesting pieces of information – e.g., most cancellations occur at step #2 of the installer, the two most common categories of responses were “Installation Problem > Other” and “I’m Confused > Didn’t Understand Instructions.” While these findings are definitely useful, we found that the holy grail of this exercise turned out to be the written comments from users.
Here are some example comments:
“i already had the latest version of firefox installed! there should be something that tells you on the website that you have the latest version before the installer is downloaded”
“It keeps telling me to close firefox and it is closed..I even went to the task manager to be sure there wasn’t one running that i couldn’t see…”
“I do not have administrative rights”
“telling me i don’t have access to write to the installation directory and i don’t know what the installation directory is or how to change it”
Going through each individual comment, we decided to categorize them to see if there were any overarching experiences encountered by users. Taking the top five most common experiences, here is a breakdown of what users cited:
Clearly, there are two key issues that have been uncovered:
- Firefox not exiting/closing cleanly (e.g., a zombie process)
- Directory location issues for non-admin users
Where do we go from here?
The Firefox product team is already discussing solutions to remedy these two problems. This is perhaps the best example we’ve seen of data+insights driving decisions and actions.
In addition, we’re eager to hear your thoughts. Perhaps there are some key insights we’ve missed in our analysis. There is a lot of valuable information that’s come out of this initiative and there’s probably more that we can be doing here (seeking feedback from users when they’re encountering pain points) in the future.