For the last few days the Test Pilot team at Mozilla Labs has been running a test to explore usage of the Firefox menu bar. Ever since Mosaic 1.0 web browsers have had a standard menu bar–one that has always followed the design of a standard desktop publishing application, containing top level commands like File and Edit, even those these commands are not necessarily relevant to a web browser.
In order to streamline the Firefox user interface, and to match the overall interactive design of Windows 7, the Firefox UX team is exploring collapsing the menu bar into a single “application button” when Firefox is running on a modern version of Windows.
This menu item usage study will help guide the UX team as they create a fully optimized design by answering 3 questions.
- Which menu items are the most commonly used?
- Which menu items are the least commonly used?
- How long do users spend exploring the menu bar contents before selecting each particular menu item?
In this post, we will discuss some preliminary findings regarding the first 2 of these 3 questions. Look for further analysis and a discussion of the 3rd question in our next post!
The most obvious way to determine the most and least commonly used menu items is to simply aggregate the total number of menu item clicks for all users.
This graph shows just that, presenting each menu item’s relative use for all UI methods (both mouse and keyboard shortcuts). Even from this simple analysis, we can see some justification for a condensed toolbar as many of the items are used very infrequently compared to the other menu items. For example, the menu items from “Page Setup” to “Character Encoding/UTF-16″ each make up less than 0.01% of the total menu bar clicks.
While looking at the total number of item clicks can be informative, since menu bars are designed for mouse use, it is more relevant to look at item usage for just the mouse UI method (excluding keyboard shortcuts).
Examining the data in this way presents a slightly different picture: the top 5 most commonly used menu items are now “User Bookmark Item”, “Copy”, “Paste”, “Add-Ons”, and “Back”. In addition to “Add-Ons”, “Options” and “Bookmark This Page” are newly part of the top 10, replacing “Find”, “Open Location”, and “Find Again”.
Again these changes simply result from eliminating keyboard shortcut clicks and help us distinguish between mouse driven menu items and keyboard driven items. For example, by comparing the mouse UI chart (right) with the original all UI chart (left) we can clearly see that “New Tab” and “Close Tab” are predominately driven by keyboard shortcuts (as expected) and may not be the two most critical items to a mouse oriented toolbar (as suggested by the original chart).
Another interesting approach to these questions is to group the items by menu and visualize the data in this form (again, data is just for Mouse UI).
This visualization presents information on two levels: the area of the circles are proportional to the total number of clicks for the menu group as a whole, and the slices correspond to the share of clicks for each item within the menu group. Bookmarks and Edit are by far the most utilized menus, representing over 70% of total clicks.
The high use of the bookmarks menu is somewhat surprising; an obvious problem of looking at aggregated data like this is the potential for outliers to skew the data. It will be interesting to delve into this issue more in depth and determine if the Bookmark menu (and other menus and menu items) is genuinely an important menu group for all users, or if the high usage is driven by a set of relatively few users who interact with the Bookmark menu extremely frequently.
Next time we will take our analysis further and move from answering questions about the frequency of item usage to examining how long users spend exploring the menu bar before selecting each particular menu item.
Thanks again to the Test Pilot Team and to all Test Pilot users for providing us with the data. Remember more information on Test Pilot studies can be found here. Anyone interested can also download data samples for this and other Test Pilot Studies from the website!