Survey Suggestions

blakecutler

12 responses

As with all of our feedback mechanisms, the uninstall survey is only useful as long as it provides insights and drives changes in our behavior.  While the existing survey has provided useful feedback, I think we can do better.   With this in mind, I have drafted an early mock up of a new uninstall survey.

In addition to changing the wording on question two, I have added three new questions, including:

  • Which browser(s) do you plan on using in the future? (select all that apply)
  • From which of the following sources did you seek support? (select all that apply)
  • How long have you been a Firefox user?

Over the next couple of weeks, I would like to publish the survey and iterate on its design to see what works best.   Do three questions work better than six?  Should some of the “None of the above” options be changed to “Other?”  Should we keep the open-ended comment box?

We have a number of options about how to implement the new survey.  We can either create and host the survey in-house, use a hosted survey tool, or embed a third party survey on survey.mozilla.com.  Above all, the new survey tool should:

  1. Be easy to localize
  2. Provide a geographic breakdown of responses
  3. Engender a high response rate

I think that a third party survey tool may be our best option for two reasons.   First, creating an internal tool that fills these requirement would be a significant undertaking.   Why should we reinvent the wheel when excellent (and cheap) survey tools already exist?  Second, our privacy concerns are minimal–the current survey explicitly states that all responses will be made publicly available.

The revised uninstall survey is still an early draft–the questions, length, and format are all up for change. Please leave your thoughts and suggestions about how we can improve the uninstall survey!

12 responses

  1. Jesper Kristensen wrote on :

    For the last question (about support), I think it might be interesting to know the difference between “Other” and “None”. There are obviously many other support options than the ones listed.

  2. Jesse Ruderman wrote on :

    Including the support question seems like a clever way to win some users back at the last second 😉

    I think it’s strange for one of the options to be “support.mozilla.com”. Users aren’t always aware of the hostnames of the sites they visit, and furthermore, that’s where the Help menu (another option) takes you. Maybe the options should be things like “Knowledge Base”, “IRC”, and “Live Chat”.

    In “Hard to use/confusing (menus, display, etc.)”, I don’t think the parenthetical adds much.

    The question “How long have you been a Firefox user?” might be tricky for someone who installed Firefox a month ago but barely used it. Should the question be made more precise?

  3. David Rolnitzky wrote on :

    I really like the 3 new additional questions — will be really interesting to get that data.

    One option for implementing the survey would be to use Stas’s tool (which we are planning on using for the first-run survey) instead of a third party. The upside is that it’s already developed. The only real downside is that he hasn’t yet incorporated reporting tools that are as sophisticated as some of the 3rd party solutions out there.

  4. deb wrote on :

    I think six questions may be too long — I found myself skimming by the end, and if I were just trying to uninstall my browser I doubt I would take the time to read through, answer, and submit a survey that long.

    Is there any way to just show three of the six questions randomly to each person? I’m not sure if that would make sense or work statistically, but while all six questions are interesting, I think three questions would increase the number of responses you get.

    I’m pretty sure we can avoid rolling our own survey system again…Stas’s tool did a great job for the about:mozilla survey, although I’m not sure it currently has the sort of reporting tools you might want (as David mentions).

  5. Tom Page wrote on :

    When uninstalling software I often skip these questions (they are becoming more common – e.g. Songbird). If it’s one question then fine. If it’s several then many people are likely to just close the window.

    Remember many people are not technical, and may not know their IE version, or has been said above what support.mozilla.com refers to (or Jesse, what IRC refers to, but maybe that’s okay)

  6. Harry wrote on :

    Looks like your reasons for leaving are all sticks and no carrots. People might uninstall because they have something more desirable to switch to. I can’t imagine what, though!

  7. karen wrote on :

    is there a survey on why people instal rather than why they uninstal?

  8. MissLissy wrote on :

    I’m going to have to agree with most everyone else here. I think 6 questions is too much. I think two questions at the very most would be okay, that is people might take the time to fill the questions out during the uninstall process. The two questions I find to be most important out of the ones you had listed are “Why did you uninstall Firefox?” and “Which browser(s) do you plan on using in the future?” Those two questions alone can tell us a lot of things about the user.

  9. Aman wrote on :

    4Q.Iperceptions.com and the conversation around the 4Q concept is exacly that. KISS(Keep it simple Stupid). Also check out kampyle.

  10. kkovash wrote on :

    Hi Aman — thanks for the comments. We’ve been in touch with 4Q and Kampyle about integrating their feedback mechanisms into areas of the Firefox experience (e.g., the installation process). We should have some info to share/discuss shortly.

    Ken

  11. Aman wrote on :

    How are you guys planning to analyze the text feedback received, once it grows in volume. Will someone actually read through all the comments!?

  12. kkovash wrote on :

    Hi Aman,

    that’s another great question. we haven’t yet come across a good solution for analyzing text feedback. with the soon-to-be-replaced version of our uninstall survey, we have been reading some comments by hand. fortunately, the new uninstall survey (pictured above), or anything we do in the future with Kampyle, should have a limited amount of text to analyze.

    If you have any advice for us, we’re all ears 🙂

    Ken