March, 2009

Mar 09

Integrating Ubiquity into Firefox: UI Mockups

I’ve recently been working with Blair, Mike, Aza, Atul, Jono and others to explore about how we might integrate an interface like Ubiquity into Firefox and bring the power of a language based user interface to a mainstream audience.

Here are some very initial UI explorations, the first of what will likely be many mockups.

You can view more details about the project, including how to contribute and attend the public meetings over on Blair’s blog post announcing the project.

Mar 09

Notifications and Flow

When designing interfaces we often tend to focus on aspects like discoverability and functionality. For instance, if we didn’t place the Web feed glyph in the location bar, how would users be able to discover that they could subscribe to pages?


But what if that small icon based notification of functionality is too subtle, should a Web browser consider using an info bar?


What if a browser displays a notification bar for something, and the user doesn’t notice it? Should it check to make sure?


These types of escalating levels of notification can all work to capture the user’s attention, but is there a downside? Often when designing a feature we can be overly focused on what the feature is about, instead of looking at the much larger picture of what the user is thinking and doing. For instance, let’s imagine the user opens their laptop while running to quickly look up the location of an important meeting. They are late, they are frazzled, they are out of breath, and… updating…. some of their extensions are no longer compatible, would they like to check for updates?


That’s an extreme (and perhaps a little contrived) example of the browser jumping in the way of the intently focused user, but it’s important to remember that every single question, no matter how small and insignificant, comes with a cost. The cost might occasionally be the user angrily smacking mouse buttons after their browser jumped in their way. Or the user might form a seething hatred for an anthropomorphized paper clip (that is in many ways the polar opposite of this).


In the case of the Microsoft Office Assistant, it’s easy to think about notifications as choosing to address the user without warning (they literally use a talk bubble). But really every form of notification is fundamentally about choosing to proactively address the user, regardless of how the dialog (in the literal sense) is styled. These are extreme examples of very directly interrupting the focused user. Much more often the notification is a quick question or a small icon, and the cost is just a few eye movements and brain cycles. The user notices the information presented, momentarily decides if they want to act on it, and then moves on. The problem with low impact notifications is that it’s easy to assume that there is no cost, the notification is free, only adding value.

The Cost: Breaking Cognitive Flow

The psychologist Mihály Csíkszentmihályi, defined the concept of flow, a state where the user is focused intently on their task, and the details of the interaction actually fade out of conscious thought. Probably the best way to understand the feeling of this mental state is to find yourself in it. For instance, while driving a car a long distance your mind may start to focus on more creative tasks, like talking to another passenger, or pondering where you should take your next vacation. After awhile, you suddenly realize that you haven’t been consciously thinking about the actual act of driving your car. You haven’t been driving dangerously, you’ve been stopping at stop signs and using your turn signal, it’s just that these details of the interaction were being processed outside of your conscious thought, just as tasks like walking or picking objects up are usually handled outside of conscious thought. The car becomes and extension of yourself that doesn’t need to be managed, and your mind is free to think about more interesting things. If you play a musical instrument you may notice that it can become an extension of yourself, and you don’t think about the details of playing it but simply create music. Or you might notice that you aren’t thinking about the controller when playing a video game, but the game itself. Basically your brain supports process isolation.

It Looks Like You’re Interested in Navigating to a Commonly Visited Web Site!

So what does flow specifically have to with Web browser design? If you think of the browser as your vehicle for exploring any type of information on the Web, it’s obviously very important that the tool cognitively fades away and disappears. It’s content over chrome, but in a mental cycles sense. Aspects like not visually integrating, or displaying a notification, will very easily break flow. Suddenly a few eye movements and brain cycles become a big deal, because it is pulling the user out of a whole cognitive state. The browser is no longer an extension of themselves. It has a question.

Opera, Chrome and Safari are now displaying commonly visited pages when the user opens a new tab, and Mozilla Labs is investigating similar interfaces. This is essentially a notification, without direct prompting the browser is notifying the user of what sites they regularly visit. If the user’s goal when opening a new tab is indeed to visit one of these sites, the interface is pretty convenient. However, if the user’s goal is to go somewhere else, what is the actual cost of the notification?


Unfortunately we don’t really have an accurate way of monitoring and logging the user’s train of thought to look for breaks. However, just by monitoring my own train of thought and talking to other users, I’ve found a most visited pages notification to be pretty disruptive.

Tools can influence their users just as much as users control their tools. It’s the “when all you have is a hammer everything looks like a nail” effect. So what kind of hammer is the most visited sites interface? I’m worried that it is an interface built around procrastination: it assumes the user is bored instead of curious. It reshapes the nearly infinite Web into a medium more akin to mindless channel surfing or a coffee table full of your favorite magazines. Of course instead of being presented with channels from traditional global media networks and broadcasters (IE4 channel bar), the page is highly customized to your individual tastes. But in terms of the cost of interruption, that just makes the alternate navigation paths all the more compelling. The user is focused on a particular task, and every time they create a new tab to continue working, Firefox attempts to draw them away, like a Siren armed with their full browsing history statistics.

This problem of redirection of intent even cropped up even when users were doing targeted searches with the awesome bar in Firefox 3. We received some feedback that the other results being returned were simply too distracting. Users reported that while looking for one particular page they would be interrupted as they glanced over the other search results, which also looked interesting. Sometimes they would select another result, even though it wasn’t what they were looking for. In terms of ranking results in order of what the user would likely be interested in, the algorithm was working beautifully, but in terms of keeping the user’s train of thought focused on where they initially wanted to go, the interface was failing.

As labs has been experimenting with the new tab interface, I’ve been pretty strongly advocating that they create an interface that isn’t just polite, but actually manages to not to break the user’s flow. This is obviously a rather difficult compromise to establish.

Our current attempt is something we’re calling the “cognitive shield.” This is a very light and (ideally) unobtrusive water mark on the new tab page that transitions to the most visited pages list when you mouse towards it. It’s purpose is to not be particularly interesting. Ideally it is hard to see, easy to miss, and doesn’t (on it’s own) provide any actual information. For instance, if it glowed when a page was updated (or was badged with a number of unread things like many OS X dock icons), it would be failing at it’s role of serving as cognitive shield.


There is some type of threshold for breaking the user’s cognitive flow, and we need to figure out what it is and try to stay under it.

I’m interested to see how well the new design performs, please download the extension using the latest development build of Firefox 3.1. Right now the mouse movement to trigger the transition isn’t quite right (so you get some false positives if you are going to focus the location bar), and we still need to think about the visual design of the water mark, but hopefully we’ll be able to actually thread the interactive design needle that is mouse-based browsing and simultaneously maintaining flow. The transition between the water mark and the actual interface should also probably be cleaner, it’s kind of random right now, going from a circle to a list.

Overall we are trying to create an interface where the user controls the browser, instead of the browser controlling the user. Or to put it in terms of proper etiquette:

In the vast majority of cases, the user interface should only speak when spoken to.