27
Jul 11

Progress

Tilt is getting closer and closer to a beta development stage, let’s see what’s been going on.

The most important added feature is the html preview window, currently using the Cloud9 source code editor Ace. Each time a node is double clicked, a popup panel shows up, displaying the formatted, syntax colored inner html for that particular node, along with the class and id if available. Since it would have been a tremendous work to implement a rich text edit box in WebGL, the easiest hack was to use an iframe containing a simple html document embedding the Ace editor.

This simple page is contained in an iframe defined in xul, inside a panel. On a double click event, all we needed to do was to get the iframe content document, access the <pre> editor element by id, set its inner html to the corresponding inner html of the selected node, then call a function to format and highlight the text. It’s really easy to implement such panels in Firefox, as ‘openPopup()’ and ‘closePopup()’ functions do all the heavy lifting.

The xul responsible with the iframe:

This isn’t, however, all the planned functionality for the html panel. A great and useful feature would be to actually edit and update the webpage rendering in real time as the code changes, and this is planned to be implemented in the following months. This behavior can lead to interesting functionality, and combining it with the Style Editor and Style Inspector would make a nice development/debugging environment. Thus, the next best move is to replace the Ace editor with the Orion Code Editor, actively developed in the devtools area.

To achieve this functionality, an optimized equivalent of canvas.drawWindow is needed, to minimize memory usage and seamlessly update the visualization as the DOM and style changes. A lot of progress has been done with #653656, and the first Javascript shim implementation is available. Once the specifications are consolidated and agreed upon, the MOZ_window_region_texture WebGL extension will be implemented in Tilt.

The way rendering webpages is done has changed quite a bit in the past months. The first experiments were focused on a more translucent visualization and now, all the nodes are rendered opaque. The community feedback has been great in both cases, so the best idea is to keep both rendering methods, and also adding customization functionality. The best way to achieve this is to implement a simple UI with sliders and other elements controlling hue, saturation, brightness, contrast and render mode (wireframe or opaque) for the visualization. Thus, transitioning between multiple ways of visualizing the webpage can be done seamlessly, and fine tuned values can be perfect for different goals.

The current GUI is a work in progress implementation in WebGL, as it would have been too much of a performance hit to implement simple sliders and buttons in XUL. The first mockups for the user interface have been finished:

A couple of other things are planned for the user interface, the most important ones being:

  • zoom and rotation sliders (+ auto rotation)
  • customizable thickness and spacing between nodes
  • preview pane in the lower right (showing the visible area of the webpage, like a map)
  • DOM tree structure in the lower left (along with a search field, go-to-node functionality)
  • reset, zoom in, zoom out buttons
  • menu in the upper right (including the export button)

For now, the major focus is implementing all of the UI functionality, to make Tilt more useful and friendly for debuggers both from the HTML and CSS world. It’s been quite a tremendous effort to create a special UI library and event modules in WebGL, but now that most of the elements have been finished, implementing the actual UI should be the next step.

As a final note, Tilt has been released as an alpha preview last week on hacks.mozilla.org, receiving a lot of feedback from the community. Most of the suggestions will make it into Tilt in the near future, the most interesting ones being:

  • exporting the visualization to different formats (especially 3D models – COLLADA, .OBJ), feature which would add a lot of possibilities like importing into games; the visualization will also be exported to other WebGL supporting browsers
  • modifying and updating the 3D webpage mesh on the fly (as the webpage changes, exposing CSS transforms for each node), meaning that interactive movies, animations and even “DOM art” can be done using Tilt (see this video, creating a 3D Reddit logo).
  • adding a zoom control which isolates a DOM node and it’s children only
  • display useful information on the sides of the nodes too (also a color code to make the most common html elements appear clearly)
  • implement CSS editing functionality, along with the HTML editor panel
  • using head tracking from a webcam to modify the perspective of a webpage

 


06
Jun 11

First implementation

A lot has happened in the past few days, and interesting progress has been done with Tilt, both from a developer point of view, and from the community side. But first thing’s first: Tilt is an extension, there’s working code on https://github.com/victorporof/Tilt (currently there are 19 followers and counting! lots for such a short period of time), so let’s see what it’s made of.

Project structureThe project structure is quite standard and follows the lines of any Firefox extension, with an install.rdf file containing description about the project (id, name, description, homepage url etc.) and also the target application, which is currently set to a Firefox minimum version of 4.0 and maximum version 7.0a1. I will probably change these as Firefox 7 ships in about five weeks, and it’s a pretty good idea to support even Aurora and Minefield releases. Currently, Tilt is not bootstrapped (it needs a Firefox restart when installed), but I don’t see it as a problem right now. As the project advances, I will most likely use Mozilla’s Addons SDK to achieve this.

The chrome.manifest is composed of three sections: content, locale and skin. MDN defines chrome as: “the set of user interface elements of the application window that are outside of a window’s content area. Toolbars, menu bars, progress bars, and window title bars are all examples of elements that are typically part of the chrome.” These files are all packaged in a JAR archive.

 

From the top, the content contains mostly .xul and .js files. The XUL holds the user interface and (view) and Javascript is responsible for the logic (controller) of the extension. For example, this extension will create an iframe overlaying the current browser parent node (and adding it to the stack), while this is done by accessing a menu under “Tools” -> “Tilt” -> “Initialize Tilt”.

Here is how the XUL currently looks like, also: what happens when initializing Tilt. These two files (browserOverlay.js and browserOverlay.xul) are at the highest level of the extension content. Most of the other files represent the engine to render everything (I called it jTilt for now), some helper draw functions built on top of this engine, and the visualization logic. Other helpful libraries like glMatrix are used, providing great performance, so that everything is efficient to achieve smooth rendering.

An important requirement of any Firefox extension is making it “localizable, basically the locale .dtd and .properties files containing all of the strings, messages and titles or captions of any view component. Thus, it’s easy to translate this extension in many different languages and making it scalable from this point of view.

The third and final part is the skin, with .css files defining the look and other assets (.png, .jpg, images in general) used by the extension.

I’m currently using a TextMate development environment, but there’s also a Komodo Edit project file if you prefer it instead. There’s no “right” IDE for developing Firefox extensions, and any text editor you’re most comfortable with will do. Read this for more information.

Currently, you can find the latest .xpi file in the bin folder. Drag and drop it in your Firefox window to install it. Start it by accessing “Tools” -> “Tilt” -> “Initialize Tilt”, and you should see a rotating quad, tinted with a green color and a texture loaded from chrome:

Tilt screenshot

Tilt screenshot (6 Jun, 2011)

The Bugzilla page for this project is Implement Tilt: a WebGL-based 3D visualization of a Webpage, I’ve received some quite useful feedback already. You might want to cc yourself there to receive updates on the development as soon as they appear. Of course, this blog has RSS, and I encourage you to subscribe to it.

A few other things worth mentioning are some bug pages on Bugzilla: