{"id":5078,"date":"2011-08-03T16:30:34","date_gmt":"2011-08-03T23:30:34","guid":{"rendered":"http:\/\/blog.mozilla.org\/metrics\/?p=5078"},"modified":"2019-09-18T12:05:34","modified_gmt":"2019-09-18T19:05:34","slug":"test-pilot-new-tab-study-results","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/metrics\/2011\/08\/03\/test-pilot-new-tab-study-results\/","title":{"rendered":"Test Pilot New Tab Study Results"},"content":{"rendered":"<div>\n<p>[Cross-posted at <a title=\"Mozilla User Research - Test Pilot New Tab Study Results\" href=\"http:\/\/blog.mozilla.org\/userresearch\/2011\/08\/test-pilot-new-tab-study-results\/\" target=\"_blank\" rel=\"noopener noreferrer\">Mozilla User Research<\/a>]<\/p>\n<p>The new tab page in Firefox is intentionally left blank, while some browsers present rich information on a newly opened tab.<\/p>\n<\/div>\n<div>\n<p style=\"text-align: center;\"><a href=\"http:\/\/blog.mozilla.org\/metrics\/2008\/08\/21\/a-first-look-at-the-uninstall-survey\/71-revision-8\/\" rel=\"attachment wp-att-83\"><img decoding=\"async\" loading=\"lazy\" title=\"screenshots of new tab pages in different browsers\" src=\"http:\/\/blog.mozilla.org\/userresearch\/files\/2011\/08\/temp.png\" alt=\"\" width=\"580\" height=\"222\" \/><\/a><\/p>\n<\/div>\n<div>The decision to leave new tab pages in Firefox blank was driven, in part, by a suspicion that too much information in the new tab may distract users from getting to the destination intended for the new tab. To test whether this suspicion is true and to learn more about user behavior after opening a new tab,\u00a0<a href=\"https:\/\/testpilot.mozillalabs.com\/\">Test Pilot<\/a>\u00a0recently released the\u00a0<a href=\"https:\/\/testpilot.mozillalabs.com\/testcases\/newtabpage\">New Tab Study<\/a>\u00a0and will soon release a\u00a0<a href=\"https:\/\/testpilot.mozillalabs.com\/testcases\/newtabmultivariate\">multivariate test on the new tab page<\/a>. Test Pilot is a platform collecting structured user feedback through Firefox. It currently has about 3 millions users and all the studies are opt in. You can help us better understand how people use their web browser and the Internet in order to build better products by participating studies. Test Pilot add-on is available\u00a0<a href=\"https:\/\/addons.mozilla.org\/en-US\/firefox\/addon\/test-pilot\/\">here<\/a>.The study ran for 5 days and in all, we collected 256,282 valid submissions.<\/div>\n<div>Results of the study show that on average each user daily:<\/div>\n<div>\n<ul>\n<li>opens 11 new blank tabs<\/li>\n<li>loads 7 pages<\/li>\n<li>visits 2 unique domains<\/li>\n<li>visits 2 pages in a new tab before they leave or close it<\/li>\n<\/ul>\n<p>Below are details on how a user loads a page in a new tab, their intentions when opening a new tab, and time spent on new tabs below.<\/p>\n<h1>How do users load a page in new tabs?<\/h1>\n<p style=\"text-align: center;\"><a href=\"http:\/\/blog.mozilla.org\/metrics\/2008\/08\/21\/a-first-look-at-the-uninstall-survey\/71-revision-10\/\" rel=\"attachment wp-att-85\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter\" title=\"methods of loading pages in new tabs\" src=\"http:\/\/blog.mozilla.org\/userresearch\/files\/2011\/08\/navigation_methods.png\" alt=\"\" width=\"581\" height=\"295\" \/><\/a><\/p>\n<p>\u00a0We detected 11 different methods to load a Web page in a blank tab page. Actions in the Url bar include pressing ENTER through keyboard, clicking the go button on the right side of the bar, clicking the Web page suggestions in the dropdown menu and pressing ENTER key for dropdown suggestions. Similarly 4 actions can be performed in the search bar too. Users can load a previously saved page from the bookmark bar in the toolbar or Bookmark\/History in the menu bar.<\/p>\n<p>Note:<\/p>\n<ul>\n<li>The URL bar is most used when navigating to new websites.<\/li>\n<li>The Search bar is also popular. Users rarely use search bar dropdown to look for old search terms.<\/li>\n<li>The Bookmark toolbar is used more often than the bookmark menu button.<\/li>\n<li>The History Menu button is seldom used.<\/li>\n<\/ul>\n<p>We can also classify all methods for loading web pages into either keyboard-based or mouse-based category. Generally speaking, users have a slight preference for mouse usage.<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/blog.mozilla.org\/metrics\/2008\/08\/21\/a-first-look-at-the-uninstall-survey\/71-revision-11\/\" rel=\"attachment wp-att-86\"><img decoding=\"async\" loading=\"lazy\" class=\"alignnone\" title=\"keyboard-based or mouse-based page loads in new tabs\" src=\"http:\/\/blog.mozilla.org\/userresearch\/files\/2011\/08\/method_mouse_keyboard.png\" alt=\"\" width=\"522\" height=\"204\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<h1 dir=\"ltr\">Why do users open new tabs?<\/h1>\n<h2 dir=\"ltr\">1. \u00a0\u00a0\u00a0Are they looking for a specific URL?<\/h2>\n<p>13.95% of new tabs (13,941,404) are opened while the text in the clipboard starts with \u201chttp\u201d or \u201cwww\u201d, which are very likely to be URL strings. The number is surprisingly high, although it may be caused by previous actions rather than by pasting for loading a specific URL.<\/p>\n<h2 dir=\"ltr\">2. \u00a0\u00a0\u00a0Users browse a limited set of domains, and only a small proportion of domains attract most visits<\/h2>\n<p>If we represent each user as a single point in the plot where x-axis is the number of pageloads, and y-axis is the number of unique domains visited, we can get the following graph. The dash line (diagonal) is what will happen if users always visit a different domain for each page load. When the users are not so active, pageloads less or around a few hundreds, the number of unique domains grows linearly. However, once users get to browse more, distinct domains tend to be stabilized and saturated.<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/blog.mozilla.org\/metrics\/?attachment_id=87\" rel=\"attachment wp-att-87\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter\" title=\"pageloads v.s. number of unique domains\" src=\"http:\/\/blog.mozilla.org\/userresearch\/files\/2011\/08\/domains.png\" alt=\"\" width=\"545\" height=\"419\" \/><\/a><\/p>\n<p>Globally, we check the visit frequencies of all domains, and find that globally only 17.38% domains (461,133 unique domains in total) take 80% of the total page loads (8,291,541 pageloads in total). It verifies the famous \u201c20-80\u201d law of long tail phenomena.<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/blog.mozilla.org\/metrics\/2008\/08\/21\/explaining-a-traffic-spike-at-mozillacom\/comscore_aug08\/\" rel=\"attachment wp-att-89\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter\" title=\"20-80 law\" src=\"http:\/\/blog.mozilla.org\/userresearch\/files\/2011\/08\/domains3.png\" alt=\"\" width=\"582\" height=\"351\" \/><\/a><\/p>\n<p style=\"text-align: left;\">On the individual level, we are interested in whether a single user performs the browsing movements according to the 20-80 law. For each individual, domains taking 80% of the total page visits is defined as \u201cmain domains\u201d. A user can confirm the 20-80 law if the ratio of the number of his main domains to the number of distinct domains is around 20%. According to the following fig., active users browse more web pages everyday, but the number of primary sites they go to decreases proportionally. It suggests that when users visit more sites, they prefer to go to the same sites more frequently. The result supports the existence of a speed-dial new tab page to some extent.<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/blog.mozilla.org\/metrics\/2008\/08\/21\/explaining-a-traffic-spike-at-mozillacom\/87-revision\/\" rel=\"attachment wp-att-88\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter\" title=\"ratio of number of main domains to the number of distinct domains\" src=\"http:\/\/blog.mozilla.org\/userresearch\/files\/2011\/08\/main_domains_ratio.png\" alt=\"\" width=\"576\" height=\"427\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<h1 dir=\"ltr\">Time Spent on New Tabs<\/h1>\n<\/div>\n<div>According to the study results, on average, users open\u00a0<em><strong>2 pages<\/strong><\/em>\u00a0in a new tab before they leave or close it. They load the first web page in\u00a0<strong><em>6 seconds<\/em><\/strong>\u00a0(median) after they open new tabs, and stay on the tab for\u00a0<em><strong>1 minute<\/strong><\/em>\u00a0(median) once they start browsing. The distributions of these two types of reaction timings display broad tails. The actually mean values are much higher than the medians:\u00a0users load the first web page in 45 seconds (mean) after they open new tabs, and stay on the tab for 7 minute (mean) once they start browsing, since the outliners and expected noises can vary the mean value a lot.<\/div>\n<div>Meanwhile, how users open a new tab can distinguish 2 groups of mouse-based users and keyboard-based users. The tabs invoked by \u201cPlus Button\u201d and \u201cDouble Click on TabBar\u201d represent the group of mouse-based users, and the tabs invoked by \u201cCommand+T\u201d represent the the group of keyboard-based users.\u00a0The results turn out that keyboard-based users act slightly faster than mouse-based ones, and they can stay on the same new tab a bit longer.<\/div>\n<div>\n<p style=\"text-align: center;\"><a href=\"http:\/\/blog.mozilla.org\/metrics\/2008\/08\/21\/explaining-a-traffic-spike-at-mozillacom\/comscore_aug081\/\" rel=\"attachment wp-att-91\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter\" title=\"timing difference between keyboard-based and mouse-based users\" src=\"http:\/\/blog.mozilla.org\/userresearch\/files\/2011\/08\/timing2.png\" alt=\"\" width=\"576\" height=\"341\" \/><\/a><\/p>\n<p style=\"text-align: left;\">The study is preliminary study for redesign requirement of the new tab pages in Firefox. We detect user behavior patterns of how they use the new tabs, including how they load a new page, broadness of domain visited, and the timing of different actions. In the following\u00a0<a href=\"https:\/\/testpilot.mozillalabs.com\/testcases\/newtabmultivariate\">New Tab Multivariate Test<\/a>, we will do a comparison between several designs of the new tab page, and more research questions will be answered, including whether too much information in the new tab may distract users from the original target or not.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>[Cross-posted at Mozilla User Research] The new tab page in Firefox is intentionally left blank, while some browsers present rich information on a newly opened tab. The decision to leave new tab pages in Firefox blank was driven, in part, &hellip; <a class=\"go\" href=\"https:\/\/blog.mozilla.org\/metrics\/2011\/08\/03\/test-pilot-new-tab-study-results\/\">Continue reading<\/a><\/p>\n","protected":false},"author":317,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[833],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/posts\/5078"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/users\/317"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/comments?post=5078"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/posts\/5078\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/media?parent=5078"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/categories?post=5078"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/tags?post=5078"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}