Jul 10

Visualizing the Usage of Firefox’s Main Window

Christopher Jung from the Mozilla Metrics team has a great post up detailing a recent Test Pilot study of Firefox’s main window.

Be sure to check out his post for the details.  The team has also created a really awesome interactive heat map to help us analyze the data we are collecting.

This study was similar to an early one that we ran on the traditional menu bar interface.

View the full size version of the static mockup

Jun 10

Why Tabs are on Top in Firefox 4

(best viewed in full screen)


Theora 720p

WebM 720p

In the Firefox 4 nightly builds, and in Firefox 4 Beta 1, we are changing the default tab position so that tabs are on top.  This is a preference that users can change by right clicking on any of their toolbars.  Moving the default tab position is obviously a significant and to some extent controversial change to the Firefox UI, which is why we made the video above to help explain our rationale.

Contributors who are active in the Mozilla community will know that this debate literally goes back for years.  So in some respects this video will serve as quick summary of all of the different arguments both for an against the change.  But the more interesting part isn’t about looking back, it’s about looking forward.  Recently modern browsers have been transitioning to placing tops on top, and that decision isn’t arbitrary, it isn’t about fashion.  The change to placing tabs on top isn’t about one browser versus another browser, it’s about the evolution of the Web as a platform.


Some of the Mockups Used in the Video

Tabs on bottom

Tabs on top

Conceptual Model of Tabs on Bottom

Conceptual Model of Tabs on Top

Conceptual Model of Firefox 3.5 (this mockup wasn’t used in the video, but is interesting nonetheless)

App Tab – Pandora

App Tab – Hypothetical HTML5 Map application

App Tab – Hypothetical HTML5 Spreadsheet application (spawned by one of the user’s app tabs, so it is has the privilege to provide its own interface)

Firefox Tab Based Library Window

Firefox Site Identity Based Notification System

Mouse Distance

Mouse Distance with the Window Edge

Mouse Distance, Bounce Off of the Window Edge (not included in the video)



Please use the comments below for any questions or feedback.

Apr 10

Don’t Talk about Users

… it might sound counter intuitive for a project so intently focused on usability and creating a mainstream browser, but in all seriousness I believe that Mozilla needs to (caveat: “for the most part”) stop talking about users.

Last week I attended a workshop about usability in open source projects. This gave me a chance to meet with designers from a number of other projects and organizations including Ubuntu, KDE, RedHat and OpenOffice.org, as well as members from the HCI research community. We each wrote a position paper prior to attending, and the position papers written by myself and Andy Ko from the University of Washington ended up complimenting each other well. Andy articulates the problem, and I provide a solution.

First, the problem:

100 contentious Firefox bug reports were analyzed for distinct uses of the word “user.” The results show that developers use authoritative words (such as allow, educate, and require) to describe what software does for users. Most statements involved confident speculation about what users need, expect and do, whereas a minority of statements demanded evidence for such unsubstantiated claims. The results also show that when describing users, developers describe them in general terms, rather than referring to particular uses cases or user populations. These results suggest that, at least in the broader Firefox developer community, developers rely largely on stereotype and instinct to understanding user needs. [From How do Open Source Developers Talk About Users]

Now this sounds pretty bad, we are confidently proclaiming speculative statements that lack any form of evidence. It’s perhaps not quite as dire as it initially sounds, since while we are falling back to stereotype (generally “users who are not developers”) and instinct (“we should make this easier”) at least the community is debating changes with some notion of users in mind. However, when you invoke a hypothetical user for use in debate (often someone’s mother), you can pretty much project whatever you want onto that hypothetical user in an attempt to win an argument. Stereotype and instinct make interface design seem arbitrary and subjective, they lack precision.

Here’s a proposed solution to this problem which I co-authored with Daniel Schwartz (you can also read it in the standardized academic conference format). Instead of talking about users, Mozilla should instead focus debate on core user experience principles.

Using a Distributed Heuristic Evaluation to Improve the Usability of Open Source Software

Building tools to enable a large scale distributed Heuristic Evaluation of usability issues can potentially reshape how open source communities view usability, and educate a new generation of user experience designers. We are exploring adding the ability to perform a distributed Heuristic Evaluation into the Bugzilla instance used to develop Firefox. Ideally this approach will build more of a culture around HCI principles, and will create a framework and vocabulary that will cross pollinate to other open source projects.


When contemplating how to increase the influence of user experience design in open source communities, a common approach is to attempt to “increase the involvement and visibility of UX professionals” [1]. However, this paper asks a different question: how can we convert current developers in open source projects to have a skill set equivalent to what academia or a corporation would consider to be formally trained user experience professional? The approach we propose consists of embedding HCI concepts and practices directly into the tools that control an open source community’s work flow.

Beyond controlling an open source community’s process and work flow, tools also indirectly shape the community’s values and ideals. This is important, because if social currency in the community is inherently linked to one’s ability to make the software better, the concept of better must be expanded to encompasses “easier to use.”

Quantitatively Measuring Usability

Measurements like the time it takes an application to load, the amount of memory used, or load on the cpu are all trivial to calculate, and wonderfully quantitative. One of the reasons open source communities tend to discount usability (both in practice and in artifacts like the severity descriptions in Bugzilla), is an inaccurate view that usability is an amorphous, and subjective thing that simply can’t be scientifically quantified and measured. However, measuring an application’s usability is an area where previous HCI research can make a strong and very significant contribution to open source development.

The usability inspection technique of Heuristic Evaluation, which was introduced by Jakob Nielsen [2,3,4] has emerged as one of the most common ways for professional user experience designers to evaluate the usability of a software application. Heuristic Evaluations are extremely useful because they formally quantify the usability of a software application against a set of well defined and irrefutable principles. Usability violations can be quantified individually: either an interface supports undo, or it does not, either an interface is internally consistent, or it is not, etc. Usability violations can also be quantified in aggregate: the software application currently has 731 known usability issues. Additionally, by building the tracking system on a set of agreed upon principles, much of the debate on the level of “there is no right or wrong with UI / every user is entitled to their personal opinion / all that matters is the ability to customize” which is currently found in open source software development communities may be significantly reduced. Usability heuristics will help ground these debates, just as currently no one in an open source community argues in favor of data loss, or in favor of crashing.

Injecting HCI Principles into Bugzilla

Adapting an open source community’s bug tracker to capture usability issues defined by a set of specific heuristics can reshape the way developers think about usability. Just as open source development communities currently have a shared vocabulary to describe good and bad with concepts such as performance, data loss, and crashing, usability heuristics can introduce additional concepts, like consistency, jargon, and feedback. All of these concepts, covering both the underlying implementation as well as the user interface, can now have an equal potential to impact the software application at any level of severity, from trivial to critical.

Modifying a bug tracking system to track a Heuristic Evaluation of software is reasonably straightforward. Each issue needs to be able to be associated with the specific usability heuristic being violated (for example: “using the term POSTDATA in a dialog is technical jargon“). We plan to utilize Bugzilla’s keyword functionality, similar to how current bugs can be flagged as violating implementation level heuristics, like data loss. Since the evaluations will be performed by contributors who likely will not have any additional interface design training, it is important that each heuristic is very clearly defined with specific examples and detailed explanations. Additionally, allowing contributors to view all of the bugs in the software marked as the same type of issue, both current and resolved, serves as an effective way for them to further learn about the heuristic.

We are now working to embedded to functionality needed for a distributed Heuristic Evaluation into the Bugzilla instance used to develop Firefox and Thunderbird. These specific modifications may spread to a variety of other open source projects as Bugzilla is currently used by communities including the Linux Kernel, Gnome, KDE, Apache, Open Office and Eclipse [5]. Ideally embedding HCI principles into development tools will also embed the ideals into the community. Similar to other forms of bugs, there will be a social incentive for contributors to locate and classify violations, and there will be a social incentive for other contributors to resolve them. As open source contributors travel between different communities and projects, the usability heuristics will also see a similar cross pollination between open source communities. A shared vocabulary will emerge across open source projects, allowing for clearer communication and debate.

Side Effects of Distributed Heuristic Evaluation

Today the process of Heuristic Evaluation is normally completed in corporations and academia by a small number of designers, who are extremely well practiced at identifying usability issues. However, it is worth noting two important aspects of the Heuristic Evaluation method from when it was first introduced:

Education – First, the method of Heuristic Evaluation has its roots not in the functional purpose of evaluating usability, but rather in the even more basic purpose of teaching usability. We see this in Nielsen’s 1989 SIGCHI bulletin: Teaching User Interface Design Based on Usability Engineering [2] that Heuristic Evaluation was introduced as part of the curriculum for a masters degree in Computer Science. This is still true today: the road to becoming a good user experience designer begins with mastering the identification of well defined heuristics.

Power in Numbers – The second important aspect of Heuristic Evaluations is that it was quickly found that the number of evaluators played a major role in how successful it was. Nielsen wrote in 1990 that “evaluators were mostly quite bad at doing such heuristic evaluations… they only found between 20 and 51% of the usability problems in the interfaces they evaluated. On the other hand, we could aggregate the evaluations several evaluators to a single evaluation and such aggregates do rather well” [3]. For large open source projects, Bugzilla instances often have thousands to hundreds of thousands of users.


In open source software development, the educational and distributed aspects of a Heuristic Evaluation are critically important. While the majority of open source projects currently lack user interface designers capable of performing a perfect Heuristic Evaluation in isolation, that’s irrelevant. The collaborative nature of open source projects allows for a group of contributors to effectively compete with a formerly trained user experience professional by aggregating their abilities. And similar to all of the other ways in which people contribute to open source projects, there is a mutually advantageous feedback loop: the more effort a contributor puts into improving the software, the more they are able to increase their own skill set. Hours spent performing heuristic evaluations and brainstorming ways to address usability issues will allow a new generation of user experience designers to emerge in open source communities, just as rock star software develops are currently forged there.


1. Schwartz, D. and Gunn, A. 2009. Integrating user experience into free/libre open source software: CHI 2009 special interest group. CHI EA ’09. ACM, New York, NY, 2739-2742.
2. Nielsen, J. and Molich, R. 1989. Teaching user interface design based on usability engineering. SIGCHI Bull. 21, 1 (Aug. 1989), 45-48.
3. Nielsen, J. and Molich, R. 1990. Heuristic evaluation of user interfaces. CHI ’90. ACM, New York, NY, 249-256.
4. Nielsen, J. 1994. Enhancing the explanatory power of usability heuristics. CHI ’94. ACM, New York, NY, 152-158.
5. Bugzilla Installation List, http://www.bugzilla.org/installation-list

Next Steps

Getting this set up for bugzilla.mozilla.org is now being covered in bug 561262. The initial set includes 17 core principles, which we may expand or modify over time as we apply these to all of the user experience bugs that we are currently tracking. The Firefox UX team will be pushing on these usability heuristics pretty heavily during the development of Firefox 4. Also some other significant open source projects are looking into using a similar approach for their bug databases, so hopefully this shared vocabulary for usability will begin to cross pollinate across the open source community.

Apr 10

Jakob Nielsen’s Comments about Open Source and Design

Business Week recently wrote a story about how Mozilla Labs Explores Open Source Design. Overall it was a great article, although it may have slightly misrepresented the role of voting in the design challenge process. It was probably the phrase “submissions are put to a public vote” that triggered Nielsen’s response:

“There are plenty of good ideas, but they don’t work well together with the real world,” says Jakob Nielsen, principal at the Nielsen Norman Group in Fremont, Calif. Open source encourages the addition of new solutions and ideas. In design, says Nielsen, “the brilliant idea can be the one unifying idea that can take away 10 other ideas.”

This quote has been stuck in my head for awhile, not because I specifically disagree but because it so accurately describes the paradox of doing design work in a transparent, collaborative, and open source environment. There is certainly no shortage of great ideas coming from our community, in large part thanks to the success of the Mozilla Labs Design Challenges. But as Nielsen points out, design isn’t just about brainstorming, it’s about unifying things, taking things away, and being able to say no. And this is where I think design in open source is often unfairly misconstrued to be based on voting and consensus building, effectively “design by committee.” People familiar with open source projects know that there is in fact centralized control both in terms of implementation and in terms of design.

What makes open source communities different is that contributors gain influence in a pure meritocracy. For instance, the design of the Firefox icon came out of the community, from a team of people who would later help to guide all of Mozilla’s visual identity. Concepts submitted to the recent Mozilla Design Challenge are impacting both the design of Firefox (a home tab that changes based on the user’s geolocation) and Chrome OS (proposal, submission). Our icons on Linux are created in the Tango style, because the Tango team has earned a tremendous amount of respect. Across both visual and interactive design we see a wide range of very successful contributions.

We also see a lot being turned down, patches failing review, bugs being marked wontfix, and the extremely common phrase “that would make a good extension” (translation: that is not going to make it into the product for mainstream users).

So I agree with Nielsen that “Open source encourages the addition of new solutions and ideas” and that “[designer’s create] the one unifying idea that can take away 10 other ideas.” Where I disagree is the implication that open source projects can not successfully do both.

Apr 10

Research Related to Firefox at CHI 2010

As usual there was a tremendous amount of research exploring the interface of Web browsers (and usually implemented on top of Firefox) at this year’s CHI conference in Atlanta. Here is a quick overview of some of the work. Of the 10 papers studying or modifying Firefox, the main theme this year was content sharing and rich web history, which about half of the papers explored.

1) Sharing and History

Eyebrowse: Real-Time Web Activity Sharing and Visualization
Max Van Kleek, Christina Xu, Brennan Moore, David R. Karger
In this paper,we explore the potential for letting users automatically track and selectively publish their web
browsing activities in real time on the Web. We developed a system, Eyebrowse, with three goals: first, to provide a means for individuals to better understand how they spend time on the web through visualizations and statistics; secondly, to foster social discovery and awareness through real-time web activity sharing;and finally, to build a large public corpus of web browsing trails using this method. We gathered user impressions of Eyebrowse, including perceived usefulness, feelings of self-exposure,and privacy concerns, for ascertaining ways to improve the system.

Web Site

Here’s What I Did: Sharing and Reusing Web Activity with ActionShot
Ian Li, Jeffrey Nichols, Tessa Lau, Clemens Drews, Allen Cypher
ActionShot is an integrated web browser tool that creates a fine-grained history of users’ browsing activities by continually recording their browsing actions at the level of interactions, such as button clicks and entries into form fields. ActionShot provides interfaces to facilitate browsing and searching through this history, sharing portions of the history through established social networking tools such as Facebook, and creating scripts that can be used to repeat previous interactions at a later time. ActionShot can also create short textual summaries for sequences of interactions. In this paper, we describe the ActionShot and our initial explorations of the tool through field deployments within our organization and a lab study. Overall, we found that ActionShot’s history features provide value beyond typical browser history interfaces. share interesting web pages that they found. However, these web sites only allow people to share the URLs of individual pages. If people want to share what they did on a web site, they have to write it down manually, which can be so tedious that they forego sharing the information. Social scripting services such as CoScripter [8] allow users to record and share interactions with websites, but these tools require forethought and planning to enable recording at the right time to capture a reusable script. Moreover, CoScripter’s one-to-all sharing model was found to deter many users [8], who asked for finer grained control over with whom they shared their scripts. An enhanced browser history could solve these problems, letting users easily grab sequences and share them with the desired audience.
Web Site

A Task-focused Approach to Support Sharing and Interruption Recovery in Web Browsers
Mohan Raj Rajamanickam, Billy Lam, Russell MacKenzie, Tao Su
Over the last two decades a vast number of services have moved online, and many new services have been created. Previous work shows that many users are overloaded by the number of webpages they use simultaneously. We introduce TabFour, a prototype web browser which integrates three features that address the design requirements identified in an initial design study. Webpages can be grouped into tasks, providing a unified target for resumption after an interruption. Tasks and pages can be annotated, supporting resumption after longer intervals. Finally, tasks can be shared through a simple yet novel web-service, allowing users to share groups of webpages more easily than with existing tools.

Enhancing Directed Content Sharing on the Web
Michael S. Bernstein, Adam Marcus, David R. Karger, and Robert C. Miller
To find interesting, personally relevant web content, people rely on friends and colleagues to pass links along as they encounter them. In this paper, we study and augment link-sharing via e-mail, the most popular means of sharing web content today. Armed with survey data indicating that active sharers of novel web content are often those that actively seek it out, we developed FeedMe, a plug-in for Google Reader that makes directed sharing of content a more salient part of the user experience. FeedMe recommends friends who may be interested in seeing content that the user is viewing, provides information on what the recipient has seen and how many emails they have received recently, and gives recipients the opportunity to provide lightweight feedback when they appreciate shared content. FeedMe introduces a novel design space within mixed-initiative social recommenders: friends who know the user voluntarily vet the material on the user’s behalf. We performed a two-week field experiment (N=60) and found that FeedMe made it easier and more enjoyable to share content that recipients appreciated and would not have found otherwise.
Web Site

Enabling Cross-Device Interaction With Web History
Timothy Sohn, Koichi Mori, Vidya Setlur
Internet-enabled personal devices are growing in number. As people own and use more devices, sharing information between devices becomes increasingly important. Web browsing is one of the most common tasks, thus sharing web history is a first step in supporting cross-device interaction. Current methods of sharing web history involve manual, cumbersome methods. This paper explores a system to automatically synchronize web information among a user’s personal devices, and optimize the interface to support mobile users. We describe a system that enables users to quickly find directions on their mobile phone based on past web searches, and seamlessly share favorite web pages between their personal devices.

2) Tabbed Browsing and Tasks

A Study of Tabbed Browsing Among Mozilla Firefox Users
Patrick Dubroy, Ravin Balakrishnan
We present a study which investigated how and why users of Mozilla Firefox use multiple tabs and windows during web browsing. The detailed web browsing usage of 21 participants was logged over a period of 13 to 21 days each, and was supplemented by qualitative data from diary entries and interviews. Through an examination of several measures of their tab usage, we show that our participants had a strong preference for the use of tabs rather than multiple windows. We report the reasons they cited for using tabs, and the advantages over multiple windows. We identify several common tab usage patterns which browsers could explicitly support. Finally, we look at how tab usage affects web page revisitation. Most of our participants switched tabs more often than they used the back button, making tab switching the second most important navigation mechanism in the browser, after link clicking.
Blog Post with the presentation

Multitasking Bar: Prototype and Evaluation of Introducing the Task Concept into a Browser
Qing Wang, Huiyou Chang
This paper clarifies two common patterns of multitasking on the Web, namely Multiple Tasks (MT) and Multiple Session Task (MST). To support both of these, the task concept needs to be introduced into a browser. An online pilot survey has revealed which attributes of the task concept are most significant to Web users and as a result a simple prototype, the Multitasking Bar (MB), is proposed based on these findings. The MB copes with the multitasking needs of both MT and MST in the browser by providing functions for task related Web page management and task schedule management. A two-session controlled experiment has been conducted to evaluate the MB and to compare user performance and experience when multitasking on the Web with and without support for MT and MST. Results show that support for both MST and MT significantly improves user task performance efficiency and greatly enhances the user experience when multitasking on the Web.

3) Improving the Interface

Enhancing Web Page Readability for Non-native Readers
Chen-Hsiang Yu, Robert C. Miller
Readers face many obstacles on today’s Web, including distracting content competing for the user’s attention and other factors interfering with comfortable reading. On today’s primarily English-language Web, non-native readers encounter even more problems, even if they have some fluency in English. In this paper, we focus on the presentation of content and propose a new transformation method, Jenga Format, to enhance web page readability. To evaluate the Jenga Format, we conducted a user study on 30 Asian users with moderate English fluency and the results indicated that the proposed transformation method improved reading comprehension without negatively affecting reading speed. We also describe Froggy, a Firefox extension which implements the Jenga format.
Web Site

Cookie Confusion: Do Browser Interfaces Undermine Understanding?
Aleecia M. McDonald
We performed a series of in-depth qualitative interviews with 14 subjects recruited to discuss Internet advertising. Participants held a wide range of views ranging from enthusiasm about ads that inform them of new products, to resignation that ads are “a fact of life,” to resentment of ads that they find “insulting.” We discovered that many participants have a poor understanding of how Internet advertising works, do not understand cookies, and mistakenly believe there are legal protections barring companies from sharing information they collect online. We found that participants have substantial confusion about the results of the actions they take within their browsers, and do not understand the technology they work with now. The user interface for cookie management in popular browsers may be contributing to confusion.

Remote Web Browsing via the Phone with TeleWeb
Yevgen Borodin
TeleWeb is an assistive voice-enabled application empowering users to remotely access the Web through the most ubiquitous device – the phone. The uniqueness of the technology is that it enables users to gain access to information from almost anywhere via a plain, old-fashioned telephone. TeleWeb users will be able to call their own personal numbers, authenticate themselves, and then use speech and phone key-pad to remotely browse the Web on their own PCs. TeleWeb may especially appeal to people with vision loss, as well as older adults who may find the phone interface to be more familiar and easier to use. In this paper, I describe the TeleWeb approach and the interface.


It was really great to see so many brilliant people thinking about Firefox and the Web in general. In fact there were so many papers covering topics related to Firefox that I often had to jump between session running concurrently in order to catch everything. It’s also exciting to see that even outside of the scope of Mozilla Labs there is a really broad range of research going on in the academic community exploring the potential future of the Web.

Apr 10

What I Have Against Contextual Design and Personas

Last night Boriss wrote a great post about the benefits of the contextual design process. Aspects of the contextual design process like the inquiry, work modeling and environment design are all incredibly important skills for a UX designer to have. However, I couldn’t disagree more with the premise that this process should have been applied by Lead Ubuntu designer Ivanka Majic in the design of the window manager.

Limitations of Contextual Design

Contextual design makes a lot of sense when you are considering a product for a rather specific use case. For instance, an application that helps doctors track the medical history of their patients throughout the day, or data visualizations for an investment banker. In these situations there is a lot to discover about the users, their tasks, their underlying goals, their environment, the situations where they fail, the situations where the succeed, etc. Basically, there is loads of context, and potentially a lot of context that hasn’t previously been considered in the horrible interface they might currently have.

So contextual design is really the most effective when designing to a specific audience. And this is also it’s limitation: applying it in situations where there is no specific audience is simply going through the motions and structure of a formal design process for no reason. Who are Ubuntu’s users? Humans. What do they do with their window manager, they want to manage windows.

Limitations of Personas

A similar design process is the use of Personas (not the lightweight Firefox theme, but the notion of creating a small set of hypothetical users who embody the attributes of the users you are designing for). Personas are an effective way to make sure that your product isn’t just connecting with users, but with the right kind of users. For instance, if you consider a game console there is really a pretty wide range of users who it could appeal to, from hard core gamers in their 20s shouting obscenities into their headsets to middle aged women doing Yoga on a balance board. The Xbox 360’s design appears to be very directly influenced by the three specific personas that they targeting: Striker the hard core gamer, BeatBuilder the technology trendsetter, and Velocity Girl, the casual gamer and creator of virtual user generated content. And even though they never really delivered on their persona of Velocity Girl, the vision of having that type of user was nonetheless inspired (in a wanting to implement Snow Crash kind of way). Personas help you focus your product on a specific vision when there are an infinite set of possibilities.

Designing a Better Doorknob

But let’s say we are creating the type of thing that nearly everyone on the planet might end up using at some point, like a window manager, or a Web browser, or a doorknob. Understanding the context is pretty trivial, and building personas for the door knob only really makes sense if the interactive design firm is charging by the hour (kidding).

But does this mean that we must fall into the trap of what Aza criticizes as “wishy-wasy design speak” that makes us look like we are pulling the Science of interaction design out of our collective behinds? Not at all. In fact, you can launch an entire career as a top tier usability pundit simply by quantifying how to build a better doorknob.

Designing Better Window Controls

So if getting context about users through the inquiry process and building personas doesn’t help, what aspects of interactive design should we consider when creating window controls? Off the top of my head two come to mind:

Self Describing Visual Language: the controls should visually represent what they are going to do, without the user having access to any external information like a manual, a helpful friend or memory of having used them before. This is an aspect where the the street light metaphor being used by Apple fails. The first time you look at them you realize that they are communicating using only color (and at one point a fourth control was telling the user “Purple!”). If the user ever waits for a tooltip to fall back to literal language your glyph has failed.

The new Ubuntu controls say more to the user than OS X, but they don’t fare particularly well in being self describing either, visually they seem to communicate that one control will move the window upwards, and another control will move the window downwards, sort of like a scroll bar. They don’t communicate that the window will be transformed.  The visual communication used in Windows is considerably more literal, choices include “big box” and “small thing at bottom.”

Natural Mapping: in general controls should be placed in a position relative to the effect that they will have. For instance, on a lot of stove tops if you have four burners you will also have four dials in the exact same configuration. If controls don’t have a natural mapping and the user falls back to point one (visual language) looking for pictograms of which dial relates to which burner, the natural mapping has failed. Ever turn on the wrong light switch, perhaps every time you try to turn on the correct one? Odds are it doesn’t have a natural mapping to the lights in the room.

For window controls you could make an argument that as the window maximizes it’s corners stretch outward, so the most natural mapping for the maximize control is on the outermost edge (regardless of side). I think the reason we don’t see any of the three designs going for this is that close is simply used so much more that it is worthwhile to give it more prominent visual placement when looking in from the side of the window (even if it violates what would be a better natural mapping).

So there is still plenty that the field of interaction design can offer to the design of simple everyday interactions, even if the context is “using the thing” and the persona is “a person living on planet earth.”

Mar 10

Visualizing Usage of the Firefox Menu Bar

A Heat Map of the Firefox Menu Bar

The Mozilla Labs Test Pilot team recently ran a study to explore how users interact with Firefox’s menu bar. The study is now complete and the raw data is available, so in addition to the great visualizations that they have already made, I put together a heat map:


View the full size

The data displayed is filtered to only Windows users (since we are now in the process of adapting the Firefox menu interface for Vista and 7). The data also is filtered to only display mouse interactions (keyboard shortcuts will of course remain consistent).

The hue, lightness, and saturation of each menu item are being generated on a logarithmic scale based on the usage. Here is the source file used to generate the HSL values.

In the heat map we can see that the menu items that are used vastly more than all others are the user’s bookmarks, copy and paste.


If you generate the HSL values with just a linear representation to usage the huge disparity between heavily used items and infrequently used items is of course even more pronounced in the visualization:


Using Test Pilot to Inform the Design of Firefox

The Firefox UX team and Test Pilot team put together this study and are now sorting through the data to help streamline Firefox’s menu interface for Windows Vista and Windows 7 (note that we will be keeping the traditional menu bar for XP, OS X and Linux for external consistency). This interface re-factoring is still very much in progress, but here is a quick screen grab of some initial ideas:


In this design Bookmarks are accessible both through the Firefox menu, as well as through a control placed directly on the navigation toolbar:


For the common edit commands like Undo, Cut, Copy and Paste, we are looking into possibly placing these directly to the right of the Firefox button, but only when the user has focused a text field. The benefit is that they are even easier for mouse-based users to access, while maintaining an otherwise streamlined design. The downside is a slight amount of peripheral visual noise as they appear and disappear, which we may try to mitigate with a very light visual design.

We are considering grouping extension menu items that otherwise would appear in the tools menu together into one area at the bottom of the Firefox menu to make them easier to find.

We are also exploring the use of split menu items for every item in the Firefox menu that produces a sub menu. So for instance if you click on History you will navigate your browser to view your history, and if you hover over it you will get the normal History sub-menu. In addition to streamlining some mouse interactions, this will also help us adapt the Firefox interface for touch input, where hovering isn’t possible.

This is all very much a work in progress, but hopefully we’ll be ready to start working on implementing the Firefox button for Windows nightly builds soon now that an initial design is coming together.

Feb 10

Presenting this Friday at ZURB

This Friday I’ll be giving a presentation about the design of Firefox as part of the ZURBsoapbox lecture series. If you are in the Bay Area and are interested in attending, you just need to RSVP. The talk will cover:

-The design philosophy at Mozilla
-The unique process of coordinating user experience design in an open source environment
-The future of Firefox’s user interface

More details are over at the ZURBlog. If you can’t make it, they will also be posting the talk to their podcast. I’ll be taking lots of questions, feel free to post any you have below or in their announcement.

Oct 09

Browsing Your Personal Web

I’ve been wanting to write this post for a few years now. During the development of Firefox 3, I worked on the design of the awesome bar (iteration 1, iteration 2), and on the design of Places (an internal name for History and Bookmarks in Firefox). This post details what I’ve imagined as the next chapter for both, but here’s the interesting part: it’s actually the same story.

The Fundamentals: Search vs. Browse Interactions

Searching and browsing are completely different styles of interaction. Search-based interfaces (like Google, Quicksilver, or the awesome bar), are very fast, they rely heavily on keyboard interaction, and they require you to know for the most part what it is that you are looking for. By contrast, browse-based interfaces (like Yahoo’s Directory, DMOZ, or Firefox’s Bookmarks Sidebar) are slow, rely heavily on mouse interaction, and are most effective when you only have a general idea what it is that you are looking for.

User interface designers usually differentiate between which interface, search or browse, is better suited for a particular task with the terms “recall” and “recognition,” referring to what is going on in the user’s mind. If the user is relying on recall, they are able to proactively retrieve what it is they want out of their memory. For instance, the traditional command line, is a recall, or search (with tab completion), based interface. In contrast, if the user is relying on recognition, they need to be able to see particular terms or objects on the screen before they are able to make a decision on what to do next. For instance, the standard GUI is fundamentally a recognition, or browse-based interface.

Often people focus more on the examples given than the fundamentally different aspects of the two types of interfaces, and assume that one type of UI is better than the other:

I used to use the command line, but then the GUI became popular, I hate remembering stuff, browse is better! Recognition beats recall!

I used to use the Yahoo Directory to find stuff on the Web, but then Google came out, I can quickly get to stuff, search is superior! Fast beats slow!

Or, in the case of the Firefox UI: I used to use the bookmarks sidebar to access things, but now I just use the awesome bar, it’s so much faster, search is the future!


But battling the different interface examples against each other somewhat misses the point. It isn’t about which interface, search or browse, is better than the other, it’s about which is a better match for the user’s particular task, and which is a better match for the user’s mind. So it is critical to provide the user with both, and to make sure that both are extremely well designed.

In Firefox 3 we spent most of our time focusing on improving search to navigate to specific bookmarks and history, with the introduction of changes to the location bar that a lot of people felt were awesome. The awesome bar redefined how we handled search in our UI, allowing users to match any part of the title or URL, monitoring the rate that users revisited sites, and learning to adapt to which search results they were most likely to select for particular search queries. After a little while you and your awesome bar started to know each other really well: in many cases you could type in a single letter, and it would serve up the exact page you were looking for. We pretty much nailed search.

But for user interfaces and information retrieval, getting search right is really only the first half the story.

Bookmarks == Files?

Historically Web browsers have handled bookmarks and history with an interface similar to the OS file system (and in some cases browsers have literally used the file system). Web pages are usually cast as little 16×16 files, and you can move them around using a traditional two pane window with folders. Perhaps because computer scientists were involved, there are a lot of hierarchical structures to expand broadly or deeply.




What’s really ironic is that these interfaces are always completely removed from direct act of browsing the Web. Sometimes they are in a totally separate window, sometimes they are sort of tacked onto the side of the browser in a sidebar, and sometimes they are layered over the browser in a tab. But these browse-based interfaces never really leverage the fact that the user is interacting with an application whose sole purpose is to browse things.

Browsing Your Personal Web

The Web browser UI has a lot of useful core controls for browsing information, a home control to take you back to the beginning, back and forward to explore a timeline of recent navigation, and a location bar lets you jump directly to an entirely new destination. These controls could be really useful for browsing history and bookmarks, in addition to browsing Web pages. So I believe we should fully integrate bookmarks and history into the Web browser interface.

Here is what this means specifically:

1. You can start to dive into history and bookmarks from the Home Tab


A lot of browsers have provided access to commonly visited pages. That’s great since it is a zero-configuration interface, and it’s something that we are looking into doing on our home tab as well. But in addition to viewing commonly visited pages, we want users to be able to navigate from the home tab into the bookmark folders and tags that they have specifically created. So for instance users can click through the chain of Home > Bookmarks > Recipes > Salmon with Mustard and Brown Sugar Glaze. You might be thinking, this seems slow, why not just type “salmon” into the awesome bar? Perhaps the user wasn’t able to recall (or perhaps even decide) what they wanted to cook for dinner. Or, perhaps they didn’t even realize that it was time to cook dinner until they recognized their recipes folder and it triggered the thought.

On a side note: recognition interfaces always have the potentially dangerous side effect of influencing and manipulating the user’s behavior. Perhaps the user was headed to “research” but then redirected to “recipes” instead. Or in the case of commonly visited pages being displayed on a new tab (Safari, Chrome): “I was going to get work done, but instead I went to slashdot and digg.” This is one of the reasons that we want to have a home tab. The home tab lets us have an interface that is entirely recognition based, rich, functional, but distracting as hell. And the new tab can remain purely recall based, a blank zen sea of perfect nothingness, but by itself completely useless.

2. History and Bookmarks are displayed in the content area

(Full Size)

Perhaps the most obvious way in which bookmarks and history can be fully integrated into the Web browsing experience is by rendering them not in a sidebar or separate window, but in the content area itself. The content area has a lot of nice properties: This gives us enough space to display thumbnails of bookmarks and history which allows users to quickly identify pages that they have seen before with a broad glance (we have an uncanny ability to remember colors and images and lizards). Additionally, the content area can be combined with a hierarchical sidebar for a traditional two pane organization interface.

3. These content area pages are part of your back/forward navigation queue


Another nice property of displaying bookmarks and history ranges in the content area is that these locally hosted meta-pages can enter into the user’s back and forward queue. So for instance, you can go to your folder of bills, pay the first bill, and then return back to the bills folder to pay the next one. Or if you are scanning your favorite sources of news, you can use the back button to take you to the page that contains the rest of the sites you like to read.

4. You can navigate directly to a collection of bookmarks, or a range of history…

Of all of the aspects of integration, this is the one that I am the most excited about…

Search + Browse Operations: the Best of Both Worlds

Above I stated that it isn’t about which interface is better, search or browse, but which is the most effective for the particular task. Does the user need recognition-based, but slow, or recall-based, but fast. But there is also a third option, which is to seamlessly merge the search and browse interfaces, and create a UI that is perfectly optimized with the user’s ability to recall specific or general information, while minimizing the time spent navigating!

Here is what I mean: even if you can’t recall the information you want down to the most specific level of detail, you search on the slightly more general information that you do remember, and then browse from one step out.

So, ideally I would remember all of the different companies that I pay bills to each month, and I could enter each of them into the location bar (recall, fastest).

But instead, currently I have them all in a folder, which I access by displaying the Bookmarks sidebar, expanding the “bookmarks menu” folder, finding the “apartment” folder, expanding the “bills” folder… (recognition, slow).

However by fully integrating bookmarks and history into the Web browser UI, I can use the location bar to navigate immediately to my bills folder, and then start paying my specific bills (recall, followed by recognition, fast-ish):


In this example “bills” happens to be a folder, but it could just as easily have been one of the user’s tags.

This hybrid interface also makes our history UI a lot faster. If someone is using the history sidebar, it’s usually because they can only remember partial information about what they are looking for (if they had all the information they would just use the awesome bar to navigate straight there), but they can recall the time range to search.

So instead of a small and clunky tree view, why not just let the user navigate straight to the time range they want to view:


This of course means a lot of potential history range terms and string localization, but this is a considerably more efficient way to locate a page out of your history, especially when it is combined with viewing thumbnails in the full content area and leveraging your visual memory.

Introducing the Not-So-Universal URL


Using the Web browser’s location bar to navigate your personal Web somewhat breaks the current nature of URLs. While the user is using the location bar to navigate to pages that have Resource Locators, these Resource Locators are certainly not Universal. However, I think there are two interesting caveats:

When you sync your bookmarks and history with Weave, and log in to any instance of Firefox (4?), you will now be able to navigate your personal Web. So while not completely global (that would be a massive privacy violation), these bookmark and history pages are more universal than being stored only on a single machine.

Additionally, while Weave locally encrypts all of your data before transmitting it to guarantee total privacy, it would be interesting if users could choose to publicly share a particular folder or tag of bookmarks, which would subsequently give the page an actual URL, which is of course Universal and shareable.

Very Early Design Mockups

Here is the current set of mockups to document the new functionality. This project is still in a very early conceptual phase, and as always we would love to hear your feedback.

Accessing Bookmarks with the Location Bar
Accessing History Ranges with the Location Bar
Viewing Bookmarks in the Content Area (in progress)
Viewing History in the Content Area (coming soon)

Sep 09

No Ribbon Planned for the Firefox UI

Yesterday a story circulated around the Web and Twitter that Firefox was planning to Tidy Up With Office 2007’s Ribbon. The mockup of Firefox in the article didn’t actually have a Ribbon in it, and the nice folks over at PCPro quickly added an update to the story. However word spreads fast on the Web, and there is now a good bit of confusion.

The Ribbon UI is designed to hold a large number of document creation and editing tools, Word (2007)

So to clarify: a tab based and contextual UI designed for holding a massive number of commands for document creation (a Ribbon) doesn’t actually make any sense for a Web browser, and we do not have any plans to use a Ribbon for commands in Firefox.

Firefox 4.0 Proposal

The one thing about all of this confusion that strikes me as really ironic is that the feedback coming in has been based primarily around two points:

1) A UI designed for document creation and editing (a Ribbon) makes no sense for a present day Web browser (indeed, we agree)
2) We should not get rid of the traditional menu bar on Vista and Windows 7!

While I totally agree with the first point, the ironic part is that a traditional menu bar is also a UI designed primarily for document creation and editing. For instance, the most prominent commands in the traditional menu bar are File, Edit, New, Open and Save:

Word 2.0 (1992)

Web browsers actually have a long history of illogically following the lead of Office’s UI, for instance look at the interface of Mosaic 1.0, where document creation and editing controls like File, Edit, Open and Save are displayed as being more important than core navigational controls like Back and Home:

Mosaic (1993), note the number of similarities to Word 2.0

This is where it all started, although to their credit they were actually working on a Web that had notions of document creation and editing.

Now while I’m all in favor of one day creating a full read/write Web browser where it is just as easy to create a page as it is to view one, we aren’t there yet. So interfaces based around document creation, like the old menu bar, or the new Ribbon, simply aren’t a good fit.

And that’s why we (as well as all other major browsers) are shifting towards minimizing the command UI, and having a single button that acts on the page, and a single button that acts on the browser. Additionally, both of these commands are de-emphasized by placing them far on the right side, while core navigation commands get placement in the upper left (this is of course reversed for RTL languages):

Firefox 4.0 Proposal, right side contains a minimal command UI

So I believe that for the same reasons a Ribbon makes little sense for the Firefox UI, which is primarily about tabs and navigation, a traditional menu bar (despite 16 years of history in Web browsers) also makes little sense. And now after 16 years, mainstream browsers like IE, Chrome, Safari and Firefox are exposing a UI that is designed for the specific task of browsing the Web, instead of just mindlessly mirroring Office.