Code review: browser hacking with dotjs

Jeff Griffiths

7

about:dotjs

Dotjs was originally created by Chris Wanstrath as a Google Chrome extension and as of this writing has also been ported to Firefox, Safari and Fluid. The SDK-based Firefox add-on was ported by Mozilla WebDev Ricky Rosario, and is available on AMO here and of course on github here.

The idea behind dotjs is nice and simple: you create a .js directory in your home directory and put your content scripts there. To load a script on github.com, you just name that script ‘github.com.js’.

dotjs’ code executes in a nice, simple sequence of events:

  1. a page loads in the browser
  2. dotjs compares the url of the page to a list of files on the filesystem
  3. if we have a match, the contents of the file are loaded and injected into the page

SDK API Usage

The main functionality in dotjs is implemented using page-mod:

require("page-mod").PageMod({
    include: "*",
    contentScriptFile: data.url('dotjs.js'),
    onAttach: function (worker) {
        worker.port.on('init', function(domain) {
            let host = url.URL(domain).host;
            if (!host)
                return;
 
            if (host.indexOf('www.') === 0) {
                host = host.substring(4, host.length);
            }
 
            let files = matchFile(host);
 
            // how to tell from here if we actually matched something?
            if (files.match)
                worker.port.emit('load-scripts', files);
        });
    }
});

The page-mod implementation is fairly straightforward – it attaches the ‘dotjs.js’ content script to every page loaded by Firefox. In the onAttach handler the worker is set up to listen for an init event; when it recieves this event it then gets a list of files from the matchFile function and passes the results back to the content script.

Content Script

The content script for dotjs is also pretty straightforward:

/*
 * catch the 'load-scripts' event and inject the results into the current scope.
 */
(function() {
    self.port.on("load-scripts", function(msg) {
        // bail out if we're in an iframe
        if (window.frameElement) return;
 
        if (msg.jquery) {
            eval(msg.jquery);
        }
 
        if (msg.js) {
            eval(msg.js);
        }
 
        if (msg.coffee) {
            (function() {
                eval(msg.transpiler);
            }).call(window); // coffee-script.js assumes this === window
            eval(CoffeeScript.compile(msg.coffee));
        }
 
        if (msg.css) {
            var headNode = document.querySelector('head');
            var cssNode = document.createElement('style');
            cssNode.innerHTML = msg.css;
            headNode.appendChild(cssNode);
        }
    });
 
    if (document.URL.indexOf('http') === 0) {
        self.port.emit('init', document.URL);    
    }
})();

The content script does three things:

  1. If we’re not the top-level document, bail out.
  2. If we get a load-scripts event, eval or inject whatever JS, Coffeescript or CSS passed back from the main add-on code.
  3. when the script is loaded, emit the init event to the main add-on code.

require(“chrome”)

This add-on does need chrome privileges in order to reliably find the user’s home directory and resolve the .js and .css directories in a cross-platform compatible way. All of this functionality is contained in the dotjs.js module in the add-on’s lib directory. In particular, the dotjs library uses the directory_service xpcom service to get the home directory resolve the js/css directories, as well as the SDK’s own file module to test if files exist, join paths and read file contents:

const {Cc, Ci} = require('chrome'); 
const data = require("self").data;
const file = require('file');
const dirSvc = Cc['@mozilla.org/file/directory_service;1'].getService(Ci.nsIProperties),
homeDir = dirSvc.get('Home', Ci.nsIFile).path,
jsDir = homeDir.indexOf('/') === 0 ? '.js' : 'js';
cssDir = homeDir.indexOf('/') === 0 ? '.css' : 'css';

Once we have a handle on the .js directory, we loop through the files in it to find a match based on the current web page’s url:

let filename = file.join(homeDir, jsDir, files[i]);
 
if (file.exists(filename + '.js')) {
    ret.js = file.read(filename + '.js');
    jsmatch = true;
    ret.match = true;
}

Ricky also added in Coffeescript and CSS support as a nice extra feature, so that you can write Coffeescript content scripts or CSS style-sheets and have them injected into the target site by the add-on:

if (file.exists(filename + '.coffee')) {
	ret.transpiler = data.load('coffee-script.js');
    ret.coffee = file.read(filename + '.coffee');
    jsmatch = true;
    ret.match = true;
}
 
let cssname = file.join(homeDir, cssDir, files[i]);
if (file.exists(cssname + '.css')) {
    ret.css = file.read(cssname + '.css');
    ret.match = true;
}

Performance considerations

When I read Ricky’s code I really liked it – it is simple, compact, makes sensible use of the SDK’s built-in features and design patterns, and also uses Firefox’s internal apis to get filesystem access when needed. I have also been involved with lots of discussions about various add-on memory use complaints, and wondered if I would make dotjs a little more efficient. The first real attempt at this encapsulated in pull request 19:

Optimization 1: Ricky’s original code always loaded jQuery into every page loaded in the browser. Instead, I opted to remove the jQuery dependency for the initial content script. This way jQuery is only loaded if there is a file match. [ commit ]

Optimization 2: when using page-mod, always bail out of documents that are in a frame. Social networking sites in particular are guilty of embedding lots of iframes as ‘social plugins’ which can really bog down the browser. [ commit ]

Modularity: originally the file matching code was located in main.js, but I opted to move it into dotjs.js as a separate CommonJS module, mainly so I could keep the chrome-privileged code as separate as possible. For such a small add-on as dotjs this isn’t much of a concern, but isolating chrome-privileged code into modules greatly simplifies the add-on reviewer’s job, especially for large, complex add-ons. In theory, we could even write tests for dotjs.js!

This is as far as I got. I have considered some ideas for future, mainly along these themes:

  1. We could change the logic to detect url matches without needing to inject an initial content script at all, using tabs.activeTab.url.
  2. instead of looking the scripts in ~/.js via synchronous file access, it would probably be better to get an initial list asnychronously on start-up and then get OS-level filesystem change notifications.
  3. when a script is loaded, we could cache the contents in a hash to avoid loading from the filesystem unnecessarily.

Admittedly these last optimizations are unlikely to make much difference in real-world performance unless you happen to have a really slow hard drive! I think in the meantime I’d rather spend more time hacking web pages using dotjs!

7 responses

  1. Mardeg wrote on :

    It seems when Jetpack is integrated into Firefox this would be an ideal extension to have included by default, with hopefully no less risk than pdf.js which is already in nightlies.

    1. Jeff Griffiths wrote on ::

      As Dietrich Ayala pointed out to me in an email, this add-on still has potential performance problems due to the use of synchronous file access, potentially on every page load. I think a good goal for future development would be to convert all file access to asynchronous methods, something that isnot currently supported by the SDK’s own file module.

  2. Frz wrote on :

    Aside from it being a nice sdk example i suppose, in what way is this approach better then what greasemonkey/scriptish are doing?

    1. Jeff Griffiths wrote on ::

      That’s a great point – the main difference between this add-on and greasemonkey scripts is that it is much more developer-oriented in style. I personally like the idea that my content scripts are in my home directory and I can hack away at them in my editor of choice.

  3. ochameau wrote on :

    You can avoid using require(“chrome”) by using, this brand new undocumented-unstable-low-level method:
    var homedir = require(“api-utils/system”).pathFor(“Home”);

    https://github.com/mozilla/addon-sdk/blob/master/packages/api-utils/lib/system.js#L50-67

    1. Jeff Griffiths wrote on ::

      Ah, interesting! Thanks for the tip.

  4. Nick wrote on :

    One of the best uses for things like dotjs is to update old, broken websites. Some of these sites still rely on frames for layout and management, which would mean that optimization 2 prevents us from updating these sites. Is it possible to have a way to override it so that the dotjs files are loaded for a site even if it’s in a frame?