It’s time to share the results of the Test Pilot Mobile Heatmap study launched last month.
My thanks to all the mobile Firefox users who submitted data! I’d also like to thank Gregg Lind and Blake Cutler for helping with the visualization, Lilian Weng and Raymond Etornam for contributing to the study code, and Mark Finkle and Wes Johnston for providing invaluable documentation on the workings of mobile Firefox.
66 Android phone users submitted data to this study. That’s much lower than the participation level of Test Pilot on desktop Firefox, because mobile Test Pilot is not yet bundled with the mobile Firefox beta, and because there aren’t nearly as many mobile Firefox users (yet). Standard caveats about self-selected Test Pilot samples apply. In particular, note that 12% of the users in this sample used the Console at least once. The Console is an advanced debugging feature that’s hidden by default, so it’s safe to assume anyone using it is a developer. Obviously a random sample of mobile web users would have far fewer than 12% developers.
I hope we can re-run this study in the future with more users, but for now 66 participants are enough to let us identity some interesting patterns.
The first visualization shows what percentage of the users touched each element at least once. At a glance, we can see that nearly every user swiped to the left and right sidebars, switched tabs, closed tabs, and touched the URL bar.
Back and Forward
A surprisingly low fraction of users – less than a quarter – used the forward and backward buttons in the right sidebar. This is starkly different from desktop browsing, where Back is one of the most used of all buttons. Three quarters of Android Firefox users didn’t touch these buttons even once! Note that uses of the Android hardware back button were not counted in this study. It’s likely that there is much less call for the software back button because the hardware button is more convenient.
However, I don’t think this explains everything. There’s no hardware substitute for the forward button, and yet only 24% of mobile users hit the forward button. Compare that to the desktop Firefox heatmap, done last year by the Metrics team. 60% of desktop users hit Forward at least once. That’s a pretty big difference! It makes me suspect that going back and forward in a tab’s history is just less frequently done in mobile browsing.
Undo Close Tab
Meanwhile, over a quarter of Android Firefox users hit the Undo Close Tab button at least once. The placement of this button is quite prominent, unlike on desktop Firefox where the feature is hidden away in a submenu. Maybe usage would be higher if the feature was as easy to discover on desktop Firefox as it is in the Android UI!
The second visualization shows each element’s mean number of touches per user. It’s the total number of times the element was touched by all users, divided by the number of users.
Tab switches and reloads
Something that immediately jumps out here is how many times people are switching tabs. The frequency of every other interaction is dwarfed by the number of tab switches. Because tab switches are so much more frequent than swipes to the left sidebar, it seems like a common use case is to swipe left and then do several tab switches in a row before refocusing on the content. Are users having trouble finding the tab they want, or are they just exploring?
It’s also interesting to see the relatively high frequency of reloads. On desktop Firefox, reload is used less frequently and by a lower percentage of people than the back button, but on mobile Firefox the pattern seems to be the opposite. Is it just because the back button is less important, or is there some special reason that mobile users reload a lot? Perhaps they’ve lost their signal, and are trying to refresh the page after regaining it.
By comparing the two visualizations, we can see some interesting patterns. For example, the New Tab button was used by a greater fraction of users than the Bookmark selector, but the Bookmark selector averaged more uses per user.
Most users (72%) hit the Edit URL field at least once, indicating they typed or edited a URL. But the frequency of URL-editing was low (5.1). Meanwhile, the frequency of touching the URL bar — which brings up the suggestion screen, or “awesome screen” — was more than three times higher (15.7). This means that two-thirds of the time someone opened the suggestion screen, they did not edit a URL. They might have picked a suggestion from frequently visited sites, from bookmarks, or from history.
Inputting URLs on mobile devices is still quite painful, and the data supports the idea that users do it infrequently compared to desktop — but 72% still had to go through this unpleasant process at least once during the study.
It would be interesting for a follow-up study to look at the number of page loads that result from entering a URL or doing a search on a mobile device, compared to the page loads resulting from links and bookmarks. I suspect the fraction is much lower than on desktop browsers.
Short tab histories
Because tabs were closed on average 23.5 times per user, but the new tab button was only used 1.8 times per user, we can infer that most new tabs are not opened with the new tab button. Most likely, they’re opened by touching a link in web content, or possibly a link in another app, which opens in Firefox if the user has chosen it as the default browser.
The high frequency of switching and closing, combined with the low frequency of back, forward, and new tab, suggests to me a usage pattern where most tabs are opened to a link and used only to display that single page before being closed. The common desktop-browsing pattern of surfing to many sites in a single tab — building up a tab history that makes the back and forward buttons useful — may be a rarity in mobile browsing sessions.
What else do you notice in these heatmaps? What would be interesting to investigate in a follow-up study?