How Users Open New Empty Tabs

blakecutler

17 responses

Summary: In a survey of 966 visitors to the Update page, 47.8% open new empty tabs from the new tab button, 30.4% from the file menu, and 21.8% from a keyboard shortcut.

In addition to running A/B and multivariate tests, we have begun supplementing our quantitative analysis with more qualitative feedback. Historically, many of our most impactful insights have come from survey data.

With this in mind, we recently launched a usability survey on the Firefox Update page page. Our goal? Understand how users open new empty tabs.

The Update page is an ideal place to run this survey because nearly every Firefox user views the page once. Additionally, because this page receives such high traffic, we are able to expose the survey to ~1% of users and get statistically significant results within a week.

Screen shot 2010-01-12 at 11.13.48 AM

Of the 6000 visitors who saw the survey, 966 responded–resulting in response rate of 16.1%. The data breaks down as follows:

Screen shot 2010-01-12 at 10.58.20 AM

Our early results are encouraging, but we can make a number of improvements. First, we may reduce our response bias by randomizing the order in which the responses are presented. Second, we will have more representative data if we run the survey immediately after a dot release. This will be a factor if the population of users who update Firefox immediately and the population of users who update later are heterogeneous (a likely assumption). Third, if we collect a larger sample, we can segment our analysis by operating system. We can run this analysis now (using SiteSpect’s excellent web optimization tool), but our results won’t be statistically significant. And finally, we can elicit a higher response rate by testing a few different survey styles.

Update page surveys provide one additional benefit. By comparing our survey results with our Test Pilot data, we can roughly approximate the bias in the Test Pilot population. Both data sets are biased, but our survey data should be more representative because it has a lower barrier to entry and a much larger sample.

Next up? We want to determine what’s driving the unusual traffic on the First Run page. Ken already has a survey in the works.

Have a survey question that would help us build a better Firefox experience? Please leave it in comments!

17 responses

  1. Simon wrote on :

    Interesting… it would never even occur to me to use the button or menu options, yet they account for about 80% of responses. I *always* use Ctrl-T….

  2. Dan wrote on :

    I double-click the tab bar and I know lots of people that do the same. You could probably get a higher response ratio if you included that option.

  3. Pino wrote on :

    That’s a good point by Dan. There are other ways to open a new tab then the three options mentioned, which is pretty bad for a survey. You could at least add an option “other” to make it inclusive, although then maybe people without a clue will just choose that answer (those who don’t know tabs for example).
    Still, this is a great initiative. It somewhat bothers me that 30% of the users use such an inefficient method of opening tabs though. Maybe put a hint on the last page for those users saying “it is faster to click the button on the tab bar”?

  4. Simon wrote on :

    @Dan – didn’t even know that was possible, nor did any of the half-dozen people sitting around me. I’d guess that relatively few people do, since it’s something you’d only discover by accident…

  5. Brian P. wrote on :

    Another way to get a new blank tab is to right click anywhere on the tab bar and select ‘new tab’. That’s pretty discoverable since it’s the first menu entry in the context menu.

    I rarely open new -empty- tabs. And if I do it is usually by accident as a vestige of an old habit. In general I get new tabs in three ways.
    searchbar + Alt+Enter
    Awesomebar + Alt+Enter
    Middle+Click on a bookmark/feed entry/link
    But those are not empty tabs.
    But on that rare occasion that I open a new empty tab I use the new tab button.

  6. testboy wrote on :

    Why would someone open an empty tab anyway? I open new tabs by either middle clicking a bookmark or a link (same, same) or by using the search box, which is set to open results in a new tab. I really do not know, what one should want to do with an empty tab? Stare at till you see colors?

  7. Tony Mechelynck wrote on :

    @testboy: why a new empty tab? So that whatever you’re doing next (I can think of typing a URL in the Location Bar, or, in SeaMonkey, searching from the sidebar) opens in a new tab rather than replace the current tab. When I want a new (temporarily) empty tab, I usually click the button at the left end of the tab bar.

  8. Orrin wrote on :

    I used to always use a fixed New Tab button on my toolbar (I liked that better than the new way of doing it with the New Tab button on the tab bar to the right of the existing tabs because that location always changes) or Ctrl+T. Lately I’ve been using the double-click method described above because I usually have one hand on the mouse anyway.

  9. Ken Saunders wrote on :

    Ditto to Dan’s comment.

    I either use the tab bar if it isn’t full, and Ctrl+T if it is.

    There’s an add a new tab button? JK. I got rid of it when it was first implemented. I didn’t need it, plus I got tired of opening new tabs when I was just trying to close one.

    Can it be assumed that the first people to update/upgrade are more experienced and knowledgeable users so there’s a higher chance that they use methods other than the options provided in the survey and that are not so obvious to standard(?) users?

  10. Gerv wrote on :

    “This will be a factor if the population of users who update Firefox immediately and the population of users who update later are heterogeneous (a likely assumption).”

    Why is that a likely assumption? I would guess people who updated immediately are tech-savvy people who follow release announcements, whereas those who update later are those who get the update automatically, or Firefox does a regular check, or something like that.

    Gerv

  11. Benoit wrote on :

    I never open an empty tab. What I do is opening a new tab with my home page by middle-clicking on the Home button.

    (It is more accessible to me because it is always at the same place on the screen, plus there is actually something to do on that page if I don’t know the address I want to visit.)

  12. Blake Cutler wrote on :

    @Gerv – if we assume ‘tech-savvy’ users and less-savvy users interact with their browsers differently, then running the survey well after a dot release will bias our data.

  13. Jim wrote on :

    You really need an “I don’t use tabs” option. I’ve met lots of people (including my parents) who just get confused when I start talking about tabs.

  14. Ken Saunders wrote on :

    @Jim
    Tools > Options > Tabs > uncheck
    Always show the tab bar
    Open new windows in a new tab instead

    Tab Killer add-on
    https://addons.mozilla.org/en-US/firefox/addon/1938
    3.5 compatible version here
    https://addons.mozilla.org/en-US/firefox/addons/versions/1938

    Advanced tab stuff
    http://kb.mozillazine.org/About:config_entries#Toolkit.
    hit Ctrl+f (or Cmd) and enter browser. tabs

    Of course having multiple windows open is a heck of a lot more confusing than using tabs.

    Out of respect, I won’t mention her age, but I’m 42 and my Mom has been using Firefox since 1.0 (IE before that) and she has no issues with tabs.
    Just because something is new or foreign to someone, it doesn’t mean that they can’t learn. After all, my mom went from radio > TV > 8 tracks > cassettes > CD’s > computing (including the Internet) DVD’s > 1 or 2 TV channels selected by turning a knob to 200+ and multiple remotes, from crawling > walking > driving goo goo ga ga > talking, etc, etc, etc.

  15. B.J. Herbison wrote on :

    The variety of ways people open tabs (other than the three listed in the survey) is impressive.

    I use approaches I don’t see listed above: I type in the Awesome Bar (and because of Tab Mix Plus the selected page is opened in a new tab) or click on a link (my cross-domain links are opened in new tabs by default).

    (And I also use other approaches. Of the three listed in the survey, I use a keyboard shortcut most often.)

    May I suggest you add an “other” option to future surveys? I’m not sure how you can “get statistically significant results within a week” if some unknown portion of the population can’t possibly answer correctly.

  16. Andrej wrote on :

    Why not add an option of Middle Click on the empty space after the plus sign to open an empty tab, just like in Opera? It’s a lot more effective than trying to find the CTRL+T key.

  17. mellaly wrote on :

    Why is that a likely assumption? I would guess people who updated immediately are tech-savvy people who follow release announcements, whereas those who update later are those who get the update automatically, or Firefox does a regular check, or something like that.