Menu Item Usage Study: The 80-20 Rule?

mozilla

16 responses

Last post, we identified the most and least commonly used menu items. Today, we will answer another question: how many different menu items do people actually use? It’s a simple question, but one that could be critical to the proposal to condense the menu bar into a single application button.

The 80-20 rule, or the Pareto principle, suggests that the number is on the low end — around 14 items. This popular rule of thumb states that for many events, 80% of effects come from 20% of the causes. Applied to software, the common saying is that 80% of all end users generally use only 20% of an application’s features.

Let’s see if menu bar interactions follow a similar pattern:

We expected a median around 14, but the actual median is 3 — almost 5x smaller! This means that 50% of all users clicked 3 items or less during the course of the study. Moreover, only 25% of people clicked on more than 4 commands. The rule of thumb states that 80% of users use only 20% of an application’s features, but menu bar behavior is far more limited: 80% of users use only 5 menu items, or just 7% of the total number of commands!

We were also surprised by the number of people who click only one unique menu item. Considering that over 20% of people fall into this category, we want to investigate further and identify which single item these individuals are using. Identifying these items is especially important since these single commands account for 100% of these users’ menu bar interactions — imprudently removing any of these commands could severely damage the browsing experience of these users.

Conveniently, the list roughly matches the ranking of the most commonly used menu items. This means that we can continue to base menu design decisions off of this ranking, without crippling the single-item users. If the orders were radically different, we may have needed to give special consideration to items that are important to single-item users, but not commonly used otherwise.

The presence of items such as “User Bookmark Item” and “Unknown”, however, leads us to some qualification of our findings:

  • A few items, “User Bookmark Item”, “Unknown”, and “User History Item”, actually encapsulate more than one unique command.  We either did not find it useful to parse these items further (in the case of “User Bookmark Item” and “User History Item”), or were not able to (in the case of “Unknown”).
  • A few items were not tracked during the study due to a bug (Special Characters, Character Encoding/More Encodings/*, Character Encoding/Customize List, Show All History, Organize Bookmarks, Minimize, and Zoom).  This means that the number of unique items for users hitting these commands is slightly downward biased.
  • The study ran for 5 days as the goal was to learn about conventional menu bar behavior.  Over the course of a browser’s lifetime, users probably click a few more menu items, but we want to optimize the bar for normal usage, so limiting the scope to a typical work week is fairly reasonable.
  • The findings refer solely to mouse interactions — again, keyboard shortcuts will remain consistent under any design.
  • And as always, our sample is of Test Pilot users only, and is likely not representative of the population of Firefox users as a whole.

Nevertheless, we are confident in our general point that relatively few menu items account for nearly all the interactions with the menu bar. Given how many items are actually used, the menu bar is too comprehensive and takes up too much valuable real estate — in other words, a condensed menu bar seems well justified.

Thoughts? How many different menu items do you typically use?

16 responses

  1. Nathan wrote on :

    How many menu items do I use? All of them… at some point or another. A standard menu is not just about offering the most frequently used options, but also providing a standard visually discoverable location where infrequently used, but important, options can be found.

    That option you really, really, need once a year or so… it is damn good to have a standard location for it so you don’t have to jump through hoops for it. The menu bar and its fairly standard hierarchy is a piece of the computer language that users have been familiar with since 1984. And since it is visually right on the screen in plain English (or whatever their native language is, not some set of freaking obscure icons) it is easy to walk a user through selecting options from a menu, even ones that are seldom used. And keep in mind that some menu items can be more important to some people than others – perhaps only 1 out of 1000 has some special need that requires them to use one of the less common menu items every day.

    Personally, I will be unbelievably disappointed if Mozilla does remove the standard menu bar. But in all fairness it is about what is best for the majority of users, not about what I want.

    At any rate, it is good to see someone actually trying to look at the science behind it rather than “ooh, Microsoft did such and such so we have to do it too!”

  2. Paul C wrote on :

    I know I, for one, would not use the menu at all if Add-Ons and Options had keyboard shortcuts on Windows (do they?)

  3. Zoink wrote on :

    I remember several months ago everybody was worked up about the new super bar and this and that… and there was discussion of pretty much getting rid of the bookmarks system altogether.. “because people just search for everything instead of using bookmarks”

    I pointed out that I found that silly, but was assured by many FX developers.. that nobody likes bookmarks.

    I am happy to see that bookmarks are still a big deal.. now if only there would be some work to fix the manager to make it even more useful.

    I have over 1,000 bookmarks on my list.. and would like it if bookmarks were still something that the developers cared about.

  4. Té wrote on :

    What sort of effect would Menu Editor users like me have had on this study?
    For example, I have moved my Preferences and Add-ons menu items to my context menu. Are these clicks tracked just as if I would’ve accessed them under Edit and Tools?

    I typically only use Preferences, Add-ons and the menus in my statusbar.

  5. Tony Mechelynck wrote on :

    “Keyboard shortcuts will remain consistent”, sure, but how do we know that “keyboard users” have the same “favourite menus” as “mouse users”? How do we know that “imprudently removing” some menu item won’t cripple the users who triggered them by means of the keyboard (i.e., File by Alt+F, or Edit by Alt+E, etc., or the corresponding Alt+something in localized versions; then the underlined letter in the desired submenu)?

  6. Peter Lairo wrote on :

    I think the fact that this test isn’t representative of the population of Firefox users as a whole is pretty important. Normal users don’t have the advanced users’ knowledge of customizing button, keyboard shortcuts, mouse gestures add-on, etc., and are thus much more dependent on the menu.

    I suggest to consider being either VERY conservative about removing functions from the planned Menu Button, or hiding the Menu Bar by default and making the Menu Button (or something similar) cause the full Menu Bar to appear (it would auto-hide after a command was selected, or the user clicked somewhere else, or after x seconds).

  7. Tiago Sá wrote on :

    @ Tony
    I think when we say “keyboard shortcuts” we mean CTRL+X and whatnot, not keyboard shortcuts for the menus…

    A question I initially thought you’d bring up is how to tell the user what the keyboard shortcuts are if we won’t have the menu options as visible. For example, if you wanna select all, and you don’t know what the keyboard shortcut is, you simply go to Edit, and look up Select All, and there you have, CTRL+A. It’s very common, but there’s always variations in some software. I think it’s important to keep telling the user about each different keyboard shortcut.

    I use an awesome extension called keyconfig, which tells me about all keyboard shortcuts (that come by default or on other extensions) and let’s me customize them. I think it’s a very important feature (to be able to costumize keyboard shortcuts) and many productivity applications have it. Web browsers do not, and I really see no reason why, besides the developers thinking it’s not relevant. It’s relevant, it’s very relevant, and it would weight very little if it was bundled with Firefox. AND it would be a good way of solving the problem of how to tell the user what each keyboard shortcut does.

    @ Peter
    I think you’re overestimating the sample that uses test pilot. It’s certainly not representative, but it’s not that far from the truth. There are, I’m sure, many Testpilot users that have very little understanding of IT, and the we can match what each user uses to the level of expertize he has selected for himself in the survey. At least we can see a trend and THEN take conclusions. Now, just assuming that those who don’t use test pilots spend their life going to the menus left and right, that’s just wrong. Alas, in my experience, unexperienced users often don’t use the menus at all! They certainly tend not to use keyboard shortcuts, but using the menus is even less frequent for unexperienced users. If it’s not in the context menu, it’s most likely out of their perception of reality.

    Print? No idea, I’ll just copy the text I want, paste it somewhere with a toolbar button that looks like a printer (word processing software) and get on with it.

    Send link? Work offline? Find? Character encoding? Zoom? Fullscreen? Even options and other stuff! No idea what it is. That’s how the unexperienced user sees the world. Menus are, in my experience, something frightening and messy. Lots of text and whatnot. A floppy disc, a printer or a scissor, people can understand that. But the menus are for experts 😛

    It’s no wonder the new tab option is used so much, since, at least in the past, there was no new tab button outside the menus in Firefox… Which was outstandingly bad. Even today, I still have the new tab button immediately to the left of the location bar, and I don’t want it anywhere else. Out of habit.

  8. Raspy wrote on :

    I think that the most important thing is when the time comes to inform most users that you are indeed changing the layout and the position of things.

    You should start working on a little tutorial that will guide the user and explain to them where is every element located and what had actually changed. this should alleviate most of the shock when the end user upgrades to a version that has a different layout to it.

    Also people can adapt. There is a time of whining that inevitably happens when every software layout and social site undertakes a major layout change. But after a while there is not many complaints. This is of course if the layout change is good.

    So make a good layout. Test it beforehand with a lot of users. Make sure that you get as many people as you can and listen to the feedback. Change things accordingly. And in the end most people will accept the change.

    And for the record I use like probably under 5 items with regularity and under 10 across a large amount of time. The thing to note is that I do have a mouse-gestures addon that helps me reach a harder to reach function.

    That is under History – Recently closed tabs. I think that this should be a more prominent menu. It is really useful at times.

  9. SilverWave wrote on :

    I have been using “Hide Caption” to get a feel for this new menu-bar-less world.

    I have the following buttons on the address bar:
    Open Extension Manager Button
    Open Options Window Button
    Clear Recent History
    Zoom In
    Zoom Out
    Clear Recent History.

    + A Toggle Menu Button.

    Its amazing how little I have to show the menu.

    May be once every couple of days.

  10. jmdesp wrote on :

    The results of your study are probably getting similar to the results that lead Microsoft to decide to auto-hide many of the menu entry in the Office programs (or inside the start menu). I hated that decision, and I’m not the only one). Be careful about what the *logical* consequences of your finding should be.

  11. mindwarp wrote on :

    How many do I typically use? Let’s see: Save Page As, Exit, File | Restart (since I have Mr. Tech Toolkit installed – I also use extensively Tools | Extension Options from that extension), Toolbars, Downloads, Add-ons, About Firefox, Check for Updates, Bookmark this Page (and, if you’re talking about saving UI space, I disable the Bookmark Toolbar and use the menu or the Awesomebar for finding my ana-retentively organized stuff). That said, I have quite a few extra menu options under Tools from extensions that I use when needed, plus some sidebars that I also use when needed. I’m old school, and not ashamed to admit it, as I use keyboard shortcuts and menu options instead of toolbar buttons in most programs.

    I only have one request: if you want to go this route for the new menu button, that’s fine, but leave the menu bar organized as it is for those of us who will enable that and use it instead of the menu button when that lands. Menu buttons/ribbons are one of the UI changes that led me away from software that was preinstalled on my Win7 machine when I got it (goodbye Office, Wordpad, and the Win7 menu, and hello OpenOffice and Classic Start Menu, which did give me back my beloved and more sensible to me classic Windows Explorer interface as well). Definitely extensively document the changes – I know that when this goes into a mainstream Firefox release, I will have two users to instruct in it if they want to go this route, so I’ll need to know it, IF they don’t have me just turn the menu bar on again (considering everyone else in my household, who are average users and are used to the classic UI which feels more logical, there’s a good chance of this happening).

  12. J. Couprie wrote on :

    I think that you are going in a wrong directions and you’ll get complaints and not praise and just loose users to other browsers by this idea of simplification of the UI.

    There is some interest in increasing user area : this means for me less page-down (or turns of the wheel on the mouse) but I may loose more by the difficulty to reach a command available.
    The true direction is for me usability by customization to my needs and adaptation to them : I don’t think that you can fulfill the needs of everyone by a standard interface.

    One of my preferred shareware programs as regards User Interface and usability is Total (previously Window) commander a file-manager with the same starting idea that the old Dos Norton commander. It offers (from top to bottom) all the ways I know to run a command.
    1) the menu (the ribbon)
    2) a line of icons : first part standard, second part user defined (with a simple way to define a command or a program to run).
    3) a place for command line (to run against the selected file or choose the selected file).
    4) a line of PF keys with their meanings .
    Of course all these lines make the user area smaller : it is the drawback to usability.

    What is for me the ideal interface ?
    A variant of this that adapt to my learning curve and that I can customize to my needs…
    What is an entry in the menu ? A name that translate to a Label, a Function, a Subroutine internal or external to the program… I need a documentation on them and a simple way to transform this name to an icon for 2) or a PF key or keyboard shortcut for 4).
    At my first contact with the program and as long I remain a newbie, it should show 1) to 4).
    2) should show
    -in the standard part, an icon to show/hide 1), 2) or 3) followed by the icons of most commonly used commands of the menu according your survey.
    -in the user defined part nothing : I’ll add icons (coming from 1) the menu) one by one as soon I discover that I use them often.
    As my experience increase, I may define short-cuts (to 4) ) from the menu and hide progressively 3) and 4) and even 1). If an exceptional need occurs, I can always show back what I have hidden !
    I think that this freedom of customization is the only way to avoid most of the complaints (except perhaps from the developers in charge).

  13. Joshua Rodman wrote on :

    This is sad.

    Your menus aren’t all that well organized but I use the majority of them.

    New Window, New Tab, Open File, Save Page As, Print/Page setup
    In the View menu, I adjust sizes, test character encodings, change the style, and view the source.

    In the History menu, i access Show All Histor, as well as click through to recent documents.

    I don’t use bookmarks much, I toss many bookmarks in but rarely refer to them. When I do go into the bookmarks menu, I use Organize Bookmarks as often as I click on a bookmark itself. The Organize Bookmarks menu is the only way i access bookmarks (other than search via ‘awesome bar’) I have the toolbar of bookmarks disabled.

    Under Tools, I use Downloads, Add-Ons, Page Info, Error Console, various entries from addons, and have used Clear Private Data.

    I use Window to select among various browser windows, as well as manually Minimize windows.

    I admit to almost never using the Help menu, because the browser reference is nearly useless, which is kinda par for the course for gui programs.

    So in short, I use the vast majority of the menu entries you have placed in your menus. I didn’t start using them overnight. I slowly over the course of years of using firefox have learned about these features and incorporated them into my use of firefox. You may see this as an indictment, but I was able to see all of them right away, I just didn’t know why I might want to use them or what they might be for, which is a problem to any collection of actions. In short, the menu worked by bieng both reviewable and consistent. Any plan to change the menu to a simplified version or something where reviewing the possible actions is more difficult will be a giant step backwards in usability, both because it will become less discoverable, and because it will reset the knowledge that users have gained so far.

  14. Joshua Rodman wrote on :

    Further, I’d like to point out one of the most useful features in Firefox: I can configure firefox to refuse to hide the menu bar, no matter what the javascript author wants. Yes, I keep the menu bar and the navigation buttons on every single window, even those which are requested to be undecorated. That’s how important the menu bar is, It’s vastly more important than keeping a few pixels for the content.

    The content is what some designer wants to show, the menu bar is what I want to do. My ability to do what I like with that content is more important than the display of the content.

  15. stephan wrote on :

    i use firefox exactly because it has finegrained control instantly accessible. if i want to trade control for simplicity i can use chrome or epiphany. and frequency of usage does not say something about the importance of an item. for example, you needn’t set the privacy settings more than once. but it is bad design if they are hidden by misleading titles and many mouseclicks for the sake of a little bit more content.

  16. Linda Hilton wrote on :

    I have to agree. I don’t think I would ever us ad ons if their were shortcuts on the keyboard that would acocmplish the same things.