Usability Research – Spectator and Test Pilot

Last year, the metrics team launched Spectator, a Firefox extension that collects data about how Firefox is used (thanks Polvi!).  Spectator was an early iteration of a broader platform recently launched by Mozilla Labs – Test Pilot.  Test Pilot is designed to enable UX  and product folks to test very specific hypotheses and questions – all of which are aimed at improving user experience and building a better browser.


You may have noticed that Mozilla Labs implemented its very first experiment for Test Pilot earlier this month.  The goal is to answer the following two questions (or at least put some quantitative framework around the questions):

  1. What would be the best default tab behavior after users close a tab?
  2. What should be default placement when users open a new tab?

As someone who downloaded Test Pilot and participated in this study, I was impressed by a couple things.  First, the introductory page did a nice job of summarizing what was going on (first image below).  And second, at any time I could view what data was being collected about me (latter image).  At the end of the experiment, it was then my decision whether or not to submit my data.



Moving forward, the metrics team looks forward to partnering the Test Pilot team in making an impact with the resulting data.  Stay tuned!

3 responses

  1. Ken Saunders wrote on :

    I was impressed as well by the same things that you mentioned about the Tab Open/Close Study. It was an awesome example of transparency and how to build trust with a user.

    Being able to view what was being collected during the process, what was going to be submitted, having the option to opt out at any time, and providing clear documentation about the whole process made me feel very comfortable and in control.

    I’m looking forward to participating in future tests and surveys and also to see how the metrics team breaks down and presents the data from the Tab Open/Close Study.

  2. Kurt (supernova_00) wrote on :

    This test should be run again since the change in the tab opening behavior patch was checked. It seems quite a few testers were not please with not be able to change the behavior through the UI and also did not like the tab closing behavior.

  3. Leo Petr wrote on :

    I am having troubles submitting the data. I click on Submit your data => Now uploading data… => Error!

    I’m not sure what the error is.