December, 2006

Dec 06

Meeting with Tantek Çelik

Yesterday Sherman Dickman, Myk Melez and I met with Tantek Çelik and Ryan King at the Technorati office in San Francisco to discuss potential support for microformats in future versions of Firefox.

Topics Discussed:

  • They have created a Web service to convert hCard and hCalendar microformats to their binary equivalents, called X2V.
  • Demo of Operator
  • The reason Yahoo! has added microformats to their various properties is the same reason they were early supporters of RSS, because it is the right thing to do.
  • When sending microformated content to an application, we have to remember to include the URL back to where the content came from. This is important because 1) it enables the user to go full circle, and 2) it gives the content producer an additional reason to embed microformats in their Web site.
  • The user interface of microformat detection (I’ll be blogging about this soon) [update: here is the post]
  • The user interface of microformat creation (I’ll be blogging about this soon)
  • The various threats to microformats. Embrace and Extend is a strategy that emerges out of corporations that have a culture of secrecy. A corporation’s level of secrecy translates to the probability that they may attempt to Embrace and Extend microformats.
  • Adding microformats to HTML forms to enable perfect autofill.
  • Impact of microformated content on accessibility.
  • Capturing the microformats that users encounter on the Web, and then enabling users to tag this information. This would allow the user to tag people, events, locations and Web pages with the same tagging scheme. This could potentially fit into our Places strategy.
  • We should expose rel-license somewhere in the chrome. This is something that Creative Commons has been asking us to do in Firefox.
  • It turns out that the real winner in the battle between Internet Explorer and Firefox is OGIO:


Technorati Tags:

Dec 06

Microformats – Part 3: Introducing Operator

Part 0: Introduction to Microformats
Part 1: Structured Data Chaos
Part 2: The Fundamental Types

Introducing Operator, a Microformat Detection Extension for Firefox 2

Today Mozilla Labs is releasing Operator, a microformat detection extension developed by Michael Kaply at IBM. Operator demonstrates the usefulness of semantic information on the Web, in real world scenarios.


Here are some examples of things you can do with this release of Operator, and with the Web as it exists today.

Yahoo! Local Plusoperator Your Address Book

With Operator you can send the phone number of your favorite pizza place from Yahoo! Local to your address book, without having to type anything. Click through for the full image:

Plusoperator Calendar Plusoperator Maps

If you view an event at you can easily add the event to your calendar to see if you are free, or map the location of the event to see where it will take place.


Blog Post Tag
Plusoperator Flickr

Let’s say you are viewing Fred’s post about the Mozilla lanyards given out at the Firefox Summit, and you decide you want to view more photos of the Firefox Summit. With Operator, you can easily navigate from Fred’s post to the “Firefox Summit” tag on Flickr.



Plusoperator Maps

Now let’s say you want to know where the above picture was taken. Because the photograph is geotagged, you can use Operator to quickly push its’ coordinates to Google maps.


This works for any of the millions of geotagged pictures on Flickr.


Operator requires information on the Web to be encoded using microformats, and since this method for semantically encoding information is relatively new, not all sites are using microformats yet. However, Operator works great with any blog that uses rel-tag, and the sites Yahoo! Local, Flickr, and, all of which contain millions of pieces of information expressed using microformats. As more sites begin to semantically encode data with microformats, Operator will automatically work with them as well.

Operator isn’t the first microformat detection extension for Firefox, previously microformat detection in Firefox was possible with the Tails Export extension by Robert de Bruin. Operator builds on Tails Export by having a user interface that is based around actions the user can take, instead of data types. Operator also includes support for the microformats geo and rel-tag, and is compatible with Firefox 2.

After using Operator for awhile, you will find yourself quickly transferring structured data to your favorite applications without typing a single letter, you will be hoping around the Web without navigating on hyperlinks, and you will be remixing services in ways that are really useful.

The combination of microformated content on the Web and the Operator extension for Firefox results in a kind of data cross pollination that we think is very exciting.

Operator as Tool for Adding Microformated Content to Your Web Site

[Update: Additional technical information about Operator can be found on Michael Kaply’s blog]

In addition to demonstrating why microformats are important to end users, Operator can also be used as a tool for testing microformated content you have added to your Web site. Operator’s creator Michael Kaply explains:

In creating Operator my goals were to create something that was useful to microformat developers as well as users looking to explore microformats. As such, the Operator Toolbar can be displayed in two modes: Microformats mode and Actions mode.

In Microformats mode, each button on the toolbar represents a microformat. When you select an individual entry from the submenu on the button all the handlers available for that microformat are displayed. You can also change the options so that the default handler for that microformat is invoked when an individual entry is selected, as opposed to displaying all the handlers. In addition, you can choose to use a more descriptive name (for instance Contact(s) instead of hCard).


I have added some other useful features for developers. When you hold down the Shift key and select an entry in a microformats submenu, it displays the internal representation of the selected microformat item. When you hold down the Ctrl key and select an entry in a microformats submenu, it displays the source of the selected microformat item. Also, if debug mode is turned on in options, when an invalid microformat is detected, the menu text is changed to “Invalid – select for more details.” If you select the menu item, it will give you a message indicating what we believe is incorrect about the entry.

The Operator extension is also very extensible, allowing for not only the creation of additional microformats, but also the addition of handlers to existing microformats. There are sample files provided in the directory where the extension is installed for both of these cases. To use the new microformat or handler, place it in a directory called “microformats” in your profile directory.

I know I will get a few questions about some things, so I thought I’d handle them here as well.

Q: Why didn’t you use the XSLT transforms from Brian Suda to export to iCal and vCard?

A: Brian has done some great work, and I did take a look at those in the beginning but decided against them mainly because I needed more flexibility in my code to access the individual items in the microformats. Because of this, I parse the microformats myself and store the data in Javascript. Also, I wanted all the code in the extension to be original. I realize my export is not anywhere near as complete as Brian’s – that’s being worked on.

Q: Aren’t the Tails and Tails Export extensions good enough? Why do we need more extensions in the microformat space?

A: The Tails and Tails Export extensions are great and they provided my first glimpse at using microformats in the browser. I quickly discovered though that the interfaces they provided (sidebar and status bar) were not what I was looking for and it was not easy to add custom handlers and microformats. My goal with this new work was to make it easy to tailor the microformat concept for the end user as well as provide more options for developers working on microformats.

Get Operator

You can download Operator at

Next: The User Interface of Microformat Detection

Technorati Tags: , ,

Dec 06

Microformats – Part 2: The Fundamental Types

Part 0: Introduction to Microformats
Part 1: Structured Data Chaos

The Fundamental Types

If Firefox 3 is potentially going to ship with native support for microformat detection, then a very important initial question becomes “which ones?”

I spent a lot of time over the last month pondering this, and I thought about a variety of different things:

  • The basic types of things people commonly do
  • The applications people commonly use to view various types of information
  • The questions that journalists are trained to ask to get the full story
  • Which Microformats are currently popular

Here is how these four different considerations relate to each other:


Here are the three new types of information that I believe Firefox should natively detect and display, because these three types are truly fundamental:


If you want to get overly philosophical, you can even string these three types together into a sentence to describe our perception of the universe:


These three icons are displayed in “microformat green,” when static information is detected in a Web page’s HTML. But they can also represent feeds of each type of information:


While many applications support importing information, not all of them support feeds of information. For instance, even though geocasting is a cool idea, I don’t know of a single mapping application that currently supports RSS with a payload of geos. One possible way around this limitation is for Firefox to aggregate microformats sent through feeds, and then forward this information to the appropriate application on a regular interval using that application’s API.

Note that these are certainly not the final icons. If we go with this design, we may need to ship three or four copies of the location icon at different rotations for respectful localization. Also, the calendar icon in particular could use some iteration.

The Web Browser as an Information Broker

Detecting information in Web pages and handing that information off to other applications changes the role of the Web browser from being solely a HTML renderer to being an information broker. This new role for Web browsers was discussed in several sessions at the Firefox Summit in November, and was also recently blogged about by Mitchell in her blog post Follow the Data. Here is a diagram of how the various fundamental types match up against commonly used applications (click through for a larger version):


Similar to the way Firefox handles search engines, microformat detection (and the associated applications) should be designed as a completely open and extensible platform. This framework should enable contact management, calendaring, and mapping applications to easily integrate with Firefox’s microformat detection system.

While discussion of implementation isn’t really the purpose of this UI blog, I should note that the difficult part of microformat detection is not parsing the data, it is dealing with the wide range of APIs for all of the different applications on all of the different platforms that can consume this data. Thankfully we have an incredible open source community.

This interaction model is fundamentally different from the user interface of Live Clipboard, and I believe it is better for a variety of reasons, which I will blog about at some point in the future.

Next: Introducing Operator

Technorati Tags: , ,

Dec 06

Microformats – Part 1: Structured Data Chaos

Part 0: Introduction

An Explosion of RSS Icons

On his blog, Ray Ozzie has a picture of the various types of buttons that have appeared on the Web to represent RSS feeds:


Several of these are truly bizarre. There seems to be an unusually high occurrence of coffee mugs and pills. I am not really sure what either of those have to do with syndication of content, but clearly a lot of people thought they were reasonable metaphors.

This chaotic user experience was the result of Web browsers not being proactive enough at providing the user with a consistent interface for the detection of Web Feeds:


Other Types of Structured Data Showing up on the Web

RSS Feeds are not the only type of structured data Web sites are exposing. We are now seeing similar buttons showing up on sites for other types of information.

Contact Information

Social networking sites often allow you to download a contact’s information. For instance, LinkedIn:


and openBC / XING:


Sometimes people expose their contact information as structured data on their personal homepage:


And companies occasionally do this as well:



At, iCal users can add Evite events to their calendar:


At OpenTable, Outlook users can add dinner reservations to their calendar:

Opentable apparently only Mac users get to party, and only Windows users get to eat out at nice restaurants. is a bit more egalitarian, but the trade off is a complicated user interface:


Unfortunately this still leaves out a lot of users. What if you use Lightning, or Spongecell or AirSet? All of these calendars import iCal, but is the user expected to know that? Should every site create a button for every associated app? That sends us right back to the RSS icon explosion, except it will be worse, because now we are talking about every popular type of structured data on the Web.


It is incredibly common for sites to contain location information. Often these are the results of a search, finding all the doctors/stores/apartments/subway stations/etc. near a particular address. Usually the locations are hard coded to a particular map provider. Often the mapping service isn’t so great:


There are several problems with this interaction:

  • Web sites are not usually updated fast enough to keep up with the best mapping applications.
  • The user is not in control of selecting what mapping service they would like to use. Perhaps they would prefer to use Google Earth, or the Yahoo Maps Beta.
  • There is no way to push multiple locations onto the same map
  • There is no way to save the location

While location information is an extreme example, across all of the different data types, the action of coupling the data itself to a particular service provider often seriously degrades the user experience.

Microformat Buttons Enter the Mix

We are now seeing buttons emerge for microformats (microformats are described in the previous post). Some of these buttons are better than others, and overall these are more useful than not having any affordance at all. But similar to what happened with RSS, once again the Web has evolved faster than the browser, and the user experience is now a mixture of specific (but not unnecessarily applicable) application buttons, and a sea of mostly unintelligible acronyms:


This is a problem that only the browser can solve. Just like with RSS, it is time for Web browsers to provide the user with a clean, consistent, and simplistic user interface.

Next: The Fundamental Types

Technorati Tags: , ,

Dec 06

Microformats – Part 0: Introduction

Have you been over hearing people talk about microformats and thought to yourself “what are those?” In this post I provide a quick introduction, and discuss the various ways that microformats are changing the Web.


What are microformats?

Microformats can be explained in a number of ways, but the easiest way to explain them is to just show an example.

Here is my contact information in HTML:

<div class=”vcard”>
<span class=”fn”>Alex Faaborg</span>
<div class=”org”>Mozilla</div>
<div class=”adr”>
<div class=”street-address”>1981 Landings Drive, Building K</div>
<span class=”locality”>Mountain View</span>,
<span class=”region”>CA</span>,
<span class=”postal-code”>94043</span>
<span class=”country-name”>United States</span>
<div class=”tel”>617-899-5064</div>

But this isn’t just normal HTML, it is semantic HTML:

<div class=”vcard”>
<span class=”fn”>Alex Faaborg</span>
<div class=”org”>Mozilla</div>
<div class=”adr”>
<div class=”street-address”>1981 Landings Drive, Building K</div>
<span class=”locality”>Mountain View</span>,
<span class=”region”>CA</span>,
<span class=”postal-code”> 94043</span>
<span class=”country-name”>United States</span>
<div class=”tel”>617-899-5064</div>

This additional semantic information in the HTML is an example of the hCard microformat. That’s what microformats are, adding semantics to markup to take it from being machine readable to being machine understandable.

There are lots of different microformats, ranging from very fundamental types of information like contacts, locations, and events, to the slightly more domain specific, like reviews and resumes, to the very domain specific, like wines.

Why are microformats important?

Microformats are going to change (and are currently changing) the Web in a variety of different ways:

1. Aggregation Sites

Right now if you want to sell something, you go to a site like craigslist. If you want to review something, you go to a site like epinions. And if you want to manage your social network, you go to a site like Facebook. The general model is the user travels to a particular site, and then proceeds to enter data (classified add, review, list of friends) for a particular purpose. Your information is scattered all over the Web, and you have to pick which sites you want to use.

The combination of blogging and microformats is now reversing this model. Now, your information remains in your blog, and the Web sites come to you. For instance, if you want to sell something, you can blog about it using an hListing, and a site like edgeio will find it when it aggregates classified advertisements across the Web. Similarly, the microformat hReview allows the creation of review aggregation sites, and XFN (XHTML Friends Network) allows the creation of social network aggregation sites.

2. Sharing Information with a Specific Community

Let’s say you enjoy mountain biking, and would like to share various trails with other people who also enjoy mountain biking. If you posted this information to your blog, you could geocast (RSS with a payload of geo), the locations of the mountain bike trails, and other people in the community could subscribe to this feed using an application like Google Earth.

3. Targeted Search

Let’s say you are creating a web comic, and you want other people to be able to find it. By posting your comic with a microformat agreed upon by the web comic community, the rest of the community will be able to easily find your work using a search engine.

What sites are currently using microformats?

The Microformats Wiki has information about which sites are currently displaying different types of microformated content. The notable examples are Flickr, which lets users geotag photos (5 million geos), Yahoo! Local, which encodes each search result with an hCard (15 million hCards), and, with encodes events with hCalendar (not sure how many).

Why are these sites using microformats?

Honestly, I don’t know. One possibility is that they are adding microformated content to their sites because it is the right thing to do, but that is kind of like answering “which came first, the chicken or the egg?” with “the egg came first, because it was the right thing to do.” Another possibility is these sites are marking up their content using microformats as a means of making the site itself an API. But whatever their motivation is, it’s really great that they are bootstrapping innovation on the Web.

Learning more about microformats


Getting involved with the microformat community

Email list

Coming soon…

All of the user scenarios above are interesting, but they don’t necessarily impact the Web browser, which is after all the whole point of this blog. But that brings us to to the fourth way microformats are going to change the Web.

4. The Web Browser as an Information Broker (Firefox 3?)

Much in the same way that operating systems currently associate particular file types with specific applications, future Web browsers are likely going to associate semantically marked up data you encounter on the Web with specific applications, either on your system or online. This means the contact information you see on a Web site will be associated with your favorite contacts application, events will be associated with your favorite calendar application, locations will be associated with your favorite mapping application, phone numbers will be associated with your favorite VOIP application, etc.

This is going to change the way we interact with data on the Web, and it’s something that I am going to be blogging about all this week, stay tuned.

Next: Structured Data Chaos

Technorati Tags: , ,

Dec 06

Tufte and Graphical Live Titles

Yesterday Justin Dolske and I spent the day attending a course by Edward Tufte called Presenting Data and Information.

Tufte is all about the high-resolution-high-bandwidth-multi-variable-insanely-complex-data-dump. At one point he stated that since people have a 150 megapixel visual system, it is technically impossible to overload them with complexity on a computer screen, since it simply can’t hold enough information. He loves the financial section of print newspapers (which shows 50,000 numbers on a page), and he hates PowerPoint (which shows 8). While his principles of analytical design make a lot of sense for information consumption, not all of them translate well to the domain of interface design. After a day of hearing the virtues of complexity, I feel like reading John Maeda’s The Laws of Simplicity just to balance things out.

I think some of his ideas could potentially impact Firefox however, especially for building tools to enable power users in vertical markets. For instance, one of his principles of analytic design is to seamlessly “integrate words, numbers and images.” He refers to small high resolution graphics inline with text as datawords, like this: Dataword. Mixing graphics inline with text could impact how we think about Live Titles. Justin has already proposed Graphical Microsummaries, and I think this feature could really help users consume information faster.

Dec 06

The Visual Identity and User Interface of Live Titles

Today Myk and I spent some time talking about the visual identity and user interface of Live Titles. I think I’ve come up with a way to streamline the bookmark UI in Firefox and I would love to get some feedback …

Continue reading →