In February, the Test Pilot team at Mozilla Labs rolled out a test to explore usage of the Firefox menu bar . This menu item usage study aims to help guide the UX team as they create a fully optimized design by answering 3 questions.

* Which menu items are the most commonly used?
* Which menu items are the least commonly used?
* How long do users spend exploring the menu bar contents before selecting each particular menu item?

After we received the raw data, Blake Cutler and Christoper Jung from Mozilla Metrics team did some great work to understand what this data is telling us. Here are a couple of preliminary findings regarding the first 2 of these 3 questions. Take a look!

Initial Findings

The most obvious way to determine the most and least commonly used menu items is to simply aggregate the total number of menu item clicks for all users.

This graph shows just that, presenting each menu item’s relative use for all UI methods (both mouse and keyboard shortcuts). Even from this simple analysis, we can see some justification for a condensed toolbar as many of the items are used very infrequently compared to the other menu items. For example, the menu items from “Page Setup” to “Character Encoding/UTF-16″ each make up less than 0.01% of the total menu bar clicks.

While looking at the total number of item clicks can be informative, since menu bars are designed for mouse use, it is more relevant to look at item usage for just the mouse UI method (excluding keyboard shortcuts).

Examining the data in this way presents a slightly different picture: the top 5 most commonly used menu items are now “User Bookmark Item”, “Copy”, “Paste”, “Add-Ons”, and “Back”. In addition to “Add-Ons”, “Options” and “Bookmark This Page” are newly part of the top 10, replacing “Find”, “Open Location”, and “Find Again”.

Again these changes simply result from eliminating keyboard shortcut clicks and help us distinguish between mouse driven menu items and keyboard driven items. For example, by comparing the mouse UI chart (right) with the original all UI chart (left) we can clearly see that “New Tab” and “Close Tab” are predominately driven by keyboard shortcuts (as expected) and may not be the two most critical items to a mouse oriented toolbar (as suggested by the original chart).

Another interesting approach to these questions is to group the items by menu and visualize the data in this form (again, data is just for Mouse UI).

This visualization presents information on two levels: the area of the circles are proportional to the total number of clicks for the menu group as a whole, and the slices correspond to the share of clicks for each item within the menu group. Bookmarks and Edit are by far the most utilized menus, representing over 70% of total clicks.

The high use of the bookmarks menu is somewhat surprising; an obvious problem of looking at aggregated data like this is the potential for outliers to skew the data. It will be interesting to delve into this issue more in depth and determine if the Bookmark menu (and other menus and menu items) is genuinely an important menu group for all users, or if the high usage is driven by a set of relatively few users who interact with the Bookmark menu extremely frequently.

Get Involved

These are some initial findings we learned from the study. We understand that we cannot draw simple design conclusions by just looking at these numbers, that’s why we really hope to keep the discussion with you in our forum! If you have any questions or new ways of interpreting them, please let us know!

Also, if you are interested to run some numbers by yourself, here is the place to download data samples for this menu study . Don’t forget to share your findings with us by submitting your analysis to the Test Pilot website!

By the way, if you haven’t joined the Test Pilot team, now it’s the time, – we are about to upgrade the Test Pilot extension to version 1.0 beta very soon!