October, 2007


29
Oct 07

Prism Brainstorming

Now that Prism has launched, a lot of people have been suggesting features and various directions Mozilla Labs could potentially take the experiment in the future. Here is a collection of ideas that people have recently suggested, and some things I’ve been thinking about in the back of my head. Please feel free to post your own ideas in the comments. [Update: or you can discuss on a thread on the Mozilla Labs forums about ideas for future directions].

Improving the Refracting Experience

  • We could package webapp bundles as extensions, and then have Prism search for matching extensions when the user is converting a Web application to a desktop application. These extensions would include various files to customize the Web application, like a webapp.css file to style the application, greasemonkey scripts, etc. In terms of the user interface, users would just see a screen shot of what the application will look like, along with a feature list when refracting an application out of their browser. For instance, if the user is converting Google Reader out of their browser (Tools > Convert to Application), they may see one bundle that has been downloaded by 10,000 people and another that has been downloaded by 3,000 people. After looking at the screen shots and feature lists, they choose which the one they want (or none). Users could switch which bundle they are using to modify the Web application at any point after refracting as well.
  • Automatically convert the user’s favorite Web applications based on a white list of known applications and their frecency score in places? For instance, Firefox notices that the user visits 30 Boxes every day so it drops a shortcut to 30 Boxes in the Web applications folder of their Start Menu to alternatively launch the application with Prism. Since this is just a shortcut, it’s only going to take up ~1k of space on the user’s hard drive, but it still may not be a good idea since users generally have a certain chagrin for applications that proactively try to help them. There are also some privacy implications.

Content Handling

  • Allow Web applications to register with the operating system to handle certain file types. Obviously there are a lot of security concerns with this, and this would only apply to applications that the user has personally refracted, there would need to be confirmation dialog boxes before uploading content, etc.
  • Allow the user to drag and drop files onto refracted applications, or Firefox (which will load the Web app in a new tab and then upload the file)
  • Get the process of uploading files to Web applications standardized through a group like the WHATWG, and supported by a few popular Web applications, like Zoho, and Google Docs

Integration with the Desktop Environment

Windows

  • Minimize to the system tray

OS X

  • Modify the dock icon (number of unread emails, web feed messages, etc.)
  • Run as a menu bar item (we would need to convert the favicon to monochrome)

All Platforms

  • Let Web application developers specify their own high resolution application icon so we don’t have to rely on upscaling
  • Support for frameless windows (for more widgetish apps, or stylized apps like twitterrific)
  • Get a notification api standardized through a group like the WHATWG, but leave it up to the client to determine how to deliver the notification to the user (we would probably go with growl on mac, system tray dialog bubbles on Windows, custom implementation on Linux, etc.)
  • Launch application on startup

Improvements to the Web Platform (mentioned in the original announcement)

  • Support for offline applications
  • canvas3d

Partner Builds

  • Make it really easy for application developers to deploy Prism along with their webapp bundle directly to users.

24
Oct 07

Prism

Today Mozilla Labs is announcing a new experiment called Prism. Over the last few months I’ve had a chance to contribute to many of the initial user facing design aspects of Prism, and this is one of the most exciting labs projects I’ve had a chance to work on so far at Mozilla. (It took us a few months to get this experiment launched due to another little project that has been taking up most of our time). Now that we’ve announced the project and are looking for contributers, here is a quick recap of where the project currently stands in terms of user experience.

UX Goal: Integrate Web Applications into the User’s Desktop Experience

Personal computing is currently in an awkward phase of transition. Half of the applications we interact with are client side desktop applications, and half are online Web applications. This means that half of the applications, regardless of what they are, are currently trapped inside of a Web browser, like some kind of Shakespearean play within a play. Because of this, Web applications are unable to take advantage of many of the user experience benefits afforded to client side applications. Web applications get lost amongst countless tabs of documents, and are closed by accident. Web applications are surrounded with controls like back and forward buttons, and a location bar, that have nothing to do with interacting with the application itself. For many AJAX applications, these Web browser controls literally serve no purpose. By being trapped inside the browser, Web applications are unable to fully integrate with a user’s computing environment, and when accessing Web applications, users are constantly forced to think about what underlying technology the application is based on.

Prism allows users to split Web applications out of their browser, and run them directly alongside desktop applications.

Refracting440

With Prism, you can now have shortcuts to your calendar and email in your quick launch bar, even if one (or both) are actually Web applications. You can organize your favorite applications by purpose, instead of by implementation. If you happen to be away from your computer, you can access these same applications from any Web browser. And if you ever decide to switch platforms, all of the applications you run with Prism can seamlessly switch with you. Marc Andreessen once described the browser as having the power to “reduce Windows to a partially debugged set of device device drivers.” Personally I disagree, in that I think operating systems also happen to be very useful for application launching and window management.

I believe Prism is important not just for the future of the Web, but for the future of the desktop as well. The social nature of the Web has resulted in incredibly fertile ground for new applications. Over the past few years this has resulted in a renaissance of new and innovative applications emerging on the Web, and even entirely new application archetypes, like RSS readers, social networks, and micro-blogging. Some specific applications in these new genres have been ported from the Web to the desktop, but for the most part the evolution of client side applications has significantly slowed, remaining largely in the previous era of desktop publishing. Prism will allow new innovative applications on the Web to integrate into the user’s desktop experience without any additional development effort for the applications’ creators.

Identity

People who follow planet.mozilla.org will likely recognize this project by the former name “Webrunner,” which was a reference to XULRunner, which was a reference to XUL, which was a reference to Ghostbusters. So why change the name? As much as I love geeky references, (especially recursively geeky references), we have to remember that most people aren’t going to semantically disambiguate terms the same way we do. For mainstream users, “running” means faster than jogging, and similarly “execution” involves killing people. The name Prism references a real world metaphor for splitting something apart. Also, we thought the name Prism would adapt well to an icon:

Prismicon

The Prism icon was created by the Iconfactory. If you are curious, here is the creative direction they were working off of.

The Favicon Problem

One of the biggest problems facing the user experience of Prism is that we needed some way of magically scaling favicons to the size of desktop icons. Favicons are nearly always 16×16 (but technically they can be a larger png, or contain higher resolutions in an ico file), while desktop icons go all the way up to 256×256 on Vista, and 512×512 on Leopard (because, seriously, why 1-up someone when you can 2^(n+1) them).

16To128

The traditional algorithms for image upscaling like bicubic interpolation were not going to work, since this can get rather fuzzy even for small amounts of upscaling, and certainly can’t scale images from 16×16 to 256×256 or 512×512 without looking horrible. Instead, I decided to try a more artistic design that embraced the pixel, and is reminiscent of our 8-bit heritage (click through for a larger image):

Favicons

This idea was influenced by the Icon Garden that was previously on the Apple campus, but more directly by ThinkGeek’s 8-bit Tie. The upscaling implementation is all done with <canvas>, and here is an example of the output. You can view the code to see how it works. We haven’t implemented the conversion of these upscaled png files to the proper .ico and .icns formats, so the current version of Prism unfortunately isn’t shipping with support for upscaled favicons yet. If you are interested, helping us out with icon upscaling on OS X, Windows and Linux is one of the many ways you can help contribute to the project.

User Interface

Prism’s user interface for integrating web applications into the desktop isn’t completely in place yet, but here is a mockup of the current design (click through for the full image). The interface is pretty straightforward, a single dialog that is displayed either by launching prism.exe/.app, or by selecting a menu command in Firefox:

Interface

The dialog box allows users to change the application’s icon, in case they don’t happen to appreciate dynamically generated isometric pixel art. If Prism becomes popular, we will probably start to see a lot of really cool high resolution unofficial icons for Web applications appear on sites like deviantart.com. These custom icons will also be particularly useful in cases where a favicon doesn’t happen to convert to isometric pixel art very well. Favicons that use a lot of antialiasing don’t look that good when upscaled, while favicons designed for the pixel art aesthetic look great.

Next Steps

In terms of user experience improvements to Prism, I think first we should primarily focus on getting the UI for generating webapp bundles fully in place (here is the current mockup), and getting favicon upscaling implemented is also pretty key. After that, working on creating css styles to match different operating systems could help streamline the creation of application-specific webapp.css files. I am, after all, a big fan of visual integration.


10
Oct 07

The Firefox 3 Visual Refresh: System Integration

As we get closer to moving Firefox 3 from the alpha stage to the beta stage, testers of Minefield (our nightly builds), and people who follow the user experience team’s mockups on bugzilla and forums like dev.apps.firefox are starting to see some glimpses of what Firefox 3 may look like. Since I haven’t posted to my blog yet about our strategy for the visual refresh, I figured I am long due for a post about what we are trying to accomplish, why, and when it is going to land.

Visual Integration with the Platform

Several weeks ago a mockup of the new places organizer hit digg, and public reaction was essentially “Looks nice for Vista users but what about OS X…

Placesorganizer

This was subsequently followed by a mockup of Firefox 3 on OS X hitting digg, and comments once again started flying: “Yay now it matches…” The Firefox 3 theme for OS X is being developed by the immensely talented contributers Stephen Horlander and Kevin Gerich. We are also receiving a lot of feedback and advice from Aronnax of GrApple fame.

Osxtheme

You may be detecting a trend. Visual integration with Windows and OS X is our primary objective for the Firefox 3 refresh.

Fitting in to the visual appearance of the native operating system may seem like a reasonably obvious decision, but it certainly isn’t one that every cross-platform application or windowing toolkit makes. For instance RealPlayer (image) uses a custom appearance across operating systems, as do applications built using Java’s Swing windowing toolkit (image). Personally I think a unified cross platform UI results in applications that at best feel foreign everywhere, and at worst don’t even feel like real applications. Making XUL appear truly native means we have to do some extra work when new operating systems are released, like Vista and Leopard, but I think this work clearly results in a superior user experience.

With this release of Firefox we are trying to achieve an even closer level of visual integration than we have had in previous releases. For instance, we are planning on having two different sets of icons for Windows: a set for Windows XP that uses the XP color palette, and will have the correct plastic reflectance, and a set for Windows Vista that will use a larger color pallet for objects, a smaller color pallet for glyphs, and the icons will be appropriately shiny. For Firefox 2 we weren’t able to ship two icon sets, so we literally averaged the two different aesthetics and created a theme that looked sort of right on each platform. If you are interested in the design guidelines for Windows icons, they are available for XP and Vista, and here are the guidelines for different visual variables I’ve set up based on these documents for the new Firefox icons.

[Update: I made the unfortunate mistake of not mentioning Linux when first posting because we still aren’t sure what the best way to visually integrate with Linux is, given the number of different distributions. Mozilla’s lead Firefox engineer Mike Connor and our user experience lead Mike Beltzner have both written followup posts about this. Some links to how you can join the discussion about visual integration on Linux appear below].

For OS X we are following the new unified style that is used consistently throughout all applications on Leopard. Firefox will look the same on 10.4 as it does on 10.5, similar to iTunes, and iLife 08. We are also looking into a variety of other little touches to visually integrate on the Mac, like the correct appearance of source lists (we call them sidebars), a cover flow-esque styling in the add-ons manager, and transparent panel styling (sometimes called HUD window styling) for the places drop down and the new identity button in the location bar (mockup coming soon).

One reason we want to focus to this level of detail on visual integration with various platforms is because the Web browser is an incredibly central piece of the user’s operating system, and we don’t want the user’s initial reaction to be that they have modified their computer to add some type of strange, foreign application. For instance, after you install Firefox on a completely fresh installation of Windows, we want you to feel like Firefox is a very carefully crafted part of your OS. This is particularly important on the Mac, since brand new computers only contain software that has been designed by Apple. Also, since Apple does a great job of including all of the applications that regular consumers commonly need for interacting with their computer, Firefox is in many cases the only third party application users install on OS X.

Visual design also goes beyond the user’s first reaction to the software application. As you move between performing tasks on the Web and interacting with the rest of your system, we don’t want the experience to be jarring. We view visual design as a part of interactive design, in that we want Firefox to blend into the rest of your system. Not in the sense that users no longer remember that their computer didn’t initially come with Firefox, but in the sense that the visual integration aids in achieving flow.

Mozilla’s user experience team literally wants to do a better job of visually integrating with Windows than IE, and a better job of visually integrating with OS X than Safari. I don’t know if we will be able to pull that off, but that’s the goal.

Maintaining Firefox’s Identity

One of the biggest challenges with this design direction will be achieving visual integration while still making sure that Firefox maintains a unique brand and identity. Traditionally we have had nearly the same user interface on every platform, so regardless of what operating system you are looking at, you can easily recognize Firefox. Correctly matching the color and reflectance (metal, plastic, glass) of each platform is going to diminish that effect. However, we can still leverage shape for cross platform consistency. For instance, we could come up with a unique shape for how the back and forward buttons integrate with each other, and style this form to match each operating system. The layout of controls will also likely still remain consistent between platforms. We of course also still have a very recognizable application icon.

Overall however, I personally think Firefox has in the past established its identity through interactions as opposed to the visual design of the interface itself. When users described switching to Firefox 1.0, they often talked about tabs (even though the icons were fantastic). I may have a skewed perspective on this because I’m far more of an interaction designer than a visual designer.

To give a specific example of how I see the next release of Firefox establishing its identity through interactions: Firefox 3 is going to include an incredibly useful feature where you can quickly navigate to a bookmark or history item by typing part of its name, or part of the URL (like the start and the end, which seems to be the only parts we remember anyway). Ideally (Seth is still working on this), the suggestions will learn and adapt based on the results you choose, so the interface will become even more streamlined over time:

Locationbarsearch

This is the kind of feature that once you start to rely on it, you quickly think to yourself “how did I ever get by without this?” When you think about the difference between Firefox 2 and 3, or the difference between Firefox and other Web browsers, I think it is streamlined interactions like this, or one click bookmarking, that are likely to spring to mind, as opposed to the application’s unique visual style.

Specific Tasks for the Visual Refresh

Mac OS X:

  • Design a new icon set for OS X (bug 399398)
  • Implement unified toolbar (bug 303110)
  • Implement a new theme for OS X (bug 397723)

Windows:

  • Design two new icon sets for Windows (bug 399398)
  • Tweak existing Windows theme (bug 399399)

[Update: Linux Distributions:

The implementation work for OS X is already well underway. The refreshed icons will start to be dropped into nightly builds of OS X and Windows as we have them. [Update: The same will go for Linux icons once we decide on the correct direction for visual integration. ]

Firefox Icon Inventory

Firefox 3 will have about 120 different icons, appearing at a variety of different resolutions. To track all of the work we need to do for this visual refresh I’ve set up an inventory of all of the icons we need. The Web application being used to display this data is Exhibit 2.0 created by the SIMILE research group at MIT. In addition to sorting different columns and filtering search, you can also use the controls on the far right side of the window to perform faceted browsing:

Exhibit

For instance, how many high visibility icons are being introduced for the first time in Firefox 3, have a resolution of 16×16 pixels, and have myself as the user experience contact? 3! This is really a useful way to browse data across a variety of dimensions. I was planning on setting up an exhibit view of the Firefox 3 Products Requirements Document, but Mike Connor beat me to it. In both cases the data itself is stored in a Google spreadsheet, which gets syndicated to a JSON RSS Feed, and the Web app immediately reflects changes people make to the spreadsheet. You can learn more about Exhibit from this tutorial about setting one up, and this tutorial about linking one to a Google spreadsheet.

How to Contribute

Things are now underway with the new OS X theme and producing icons. If you are interested in getting involved, there are two areas where we would love for people to contribute:

1) General Feedback on both the design direction and actual work (either below or in the relevant bugs)
2) Specific ideas for theme tweaks to help Firefox fit in on various platforms. For instance, this blog post about tweaking the Firefox theme for Vista is exactly the kind of ideas we are looking for.


8
Oct 07

Firefox and Miro

While pondering the overall user experience of the internet, I find myself often thinking about how in this new era of podcasting, the experience frankly isn’t all about email clients and Web browsers any more. If you look at content and delivery, Firefox and Miro are for the most part opposites:

Contentanddelivery

The main point of interaction between the two applications is when Firefox needs to hand a podcast off to an application like Miro. While redesigning content handling for Firefox 3, at one point we were planning on inspecting RSS feeds to see the types of enclosures, and re-filtering the list of possible applications for feeds that contained podcasts and video podcasts (this was back in the day when Miro was named Democracy):

Subscribe

As part of detecting feeds that predominately contained podcasts and video podcasts, we were considering including a link to getmiro.com in the respective application drop downs in the applications prefpane:

Applicationsprefpane

The rational behind this decision was that Windows does not currently ship with an application to download podcasts and video podcasts, and if the user only has Firefox and a fresh install of Windows, this feature would otherwise be useless. We couldn’t rely on the default content handler specified by the OS.

Even though analyzing RSS feeds got cut for Firefox 3, what do people think of potentially still including a bookmark in new profiles to getmiro.com?

[Update: initial reaction to the bookmark idea seems to be unanimously negative, and in general is being viewed as selling out (assuming that it is technically possible for one non-profit to sell out to another non-profit, I’m not entirely sure how that works :). Anyway, I’m backpedaling to the idea of creating a featured Firefox extension for podcasting, which includes the original functionality, but launches when we have time to actually implement it].

Pros

  • Miro compliments the functionality of Firefox well. Firefox is all about consuming images and text, and Miro is all about consuming audio and video. With both applications, users will be able to easily consume every type of media delivered to them over the internet.
  • Miro provides a great user experience, so introducing Firefox users to Miro will improve their overall user experience with the internet, even though this now includes interactions outside of Firefox.
  • The Participatory Culture Foundation and the Mozilla Foundation have very similar goals and values (in particular manifesto points 2, 5 and 6).

Cons

  • Adding a bookmark to new profiles increases the complexity of our default UI (even if the new bookmark is only in the bookmarks menu, and not on the bookmarks toolbar).
  • Adding a bookmark takes us in the direction of the Netscape Whorebar (although this time around it is 501c3-free-and-open-Web whoring, which is perhaps a little more wholesome.)

Overall I recognize that pre-populated bookmarks constitute incredibly valuable real estate, but I think including this particular bookmark will improve Firerfox users’ experience with the internet, and it is something we should seriously consider.

[Update: I still think that Miro will improve users’ experience with the internet, but a recommended extension for podcasting will probably work better than a bookmark until we can get native support for detecting podcasts built in to Firefox].

What do people think? Feel free to comment below, or in bug 399126.