Overview
Earlier this year, I had the privilege of working on concepting the future of Firefox OS visual design with a small team. As part of that exploration, I collaborated with front-end engineer Pavel Ivanov to see what we could do with user customization. Something we’ve always wanted was to give users of all levels of comfort with technology the ability to theme their Firefox OS experience with a single action: by simply selecting an image from their library. We envisioned that we could take a user-selected photo and use it to create OS imagery such as wallpaper, as well as customize their ui colors. This tool could eventually be a responsive web app or become integrated as part of the OS settings app.
On the visual side, we were exploring a new design language inspired by the pixel, a kind of playful homage to our engineering culture. We used it to create pattern imagery that would underpin the visuals for the entire OS. This was a natural starting point for the type of imagery that we would generate from the colors extracted from an image.
Pattern Imagery from Firefox OS Visual Concept
Color Extraction
The color extraction algorithm that we used was jointly developed by Pavel and myself through several iterations (and I should note is a work in progress). Our goal was to get as close as possible to what a human would pick as the primary set of colors they see in a given photo. This isn’t as easy as it might seem at first. A simple brute force approach of dividing an image into an m x n grid and taking the average of all the pixels within each grid element can often ‘average out’ intense colors.
In the end, our approach is pretty simple, but effective:
Step 1. Sample the colors of individual pixels at regular, fairly granular intervals, but large enough to perform relatively fast.
Step 2. Rank colors by area. Humans are good at pattern recognition, and one of the primary ways we see color in images is by collapsing similar colors into essentially one color, so that instead of seeing hundreds of greys that make up a rock, we see an average flat grey. To accomplish this, we used a delta that would effectively group a set of similar colors together and used this to calculate and rank the colors according to greatest area. This delta was arrived at through trial and error on a number of images that gave the best average performance.
Step 3. Rank colors by intensity. ‘Intensity’ in this case is just a term we used to describe a particular combination of brightness (or lack of) and saturation. This would, for example, ensure that if there were small areas of very intense color, such as a tiny flower in a field of green foliage, it would get selected, just as a human would. These colors can be missed by following a brute force average approach as well as ranking by area.
Note: Step 3 is not yet fully implemented.
Patterns
To begin with, I created four patterns (generators) based on the new visual language that Pavel implemented. Initially these were hard-coded and and drawn to Canvas, but they have since been converted to rendering with SVG. While we have a very small set at the moment, the goal is to be able to rapidly expand this set of imagery and eventually open it up to contributors.
The Tool
Firefox OS Customization Demo
The tool in its’ current form can be found at http://firefoxos-mozilla.sapp.io/
Try it out for yourself!
Step 1. Feed it an image by selecting the URL field at the top of the page and enter a URL link to any image on the web. Some images that we have used to develop this tool are listed below.
http://firefoxos-mozilla.sapp.io/wallpapers/PH_CalmDock@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_Hills@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_Leaves@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_Orange@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_PurpleFlower@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_RollingHills@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_Shore@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_Snail@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_SurrealGrass@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_Towers@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_Tree@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_TreeForest@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_Wheat@2x.jpg
http://firefoxos-mozilla.sapp.io/wallpapers/PH_WhiteFlower@2x.jpg
Step 2. Select a generator (or pattern).
Step 3. Modify the colors by selecting any two colors you wish from the extracted palette to create your own image.
Step 4. Download the image by clicking on the download link. Currently, only 720×1280 images are available, which fit the Sony Z3C (our Foxfooding device), but we will be adding support for other screens in the future.
Next Steps
While we’ve taken a good first step toward a simple but effective customization tool for Firefox OS, there are a number of things we want to do with it before it’s truly useful. The next steps include converting it to a responsive web app, auto-detecting device screen size, adding new patterns and improving the color extraction algorithm.
Stay tuned!
Credits
Developer: Pavel Ivanov
Designer: Peter La
Photography: Javier Pardina (http://pardina.co/)