Building A JavaScript Library With Tests, Mocks, and CI

I talked recently at DjangoCon about how to build better Python packages. As engineers we all use each other’s open source libraries everyday so it’s nice when 1) they work, 2) they’re well documented, and 3) they’re well tested.

For a recent project I set out to build a JavaScript library for other developers to use in their web applications. It had been a while since I made a browser library in JavaScript so I took it as an opportunity to see if I could apply some Python packaging concepts to JavaScript.

This is a brief summary of some tools and patterns I settled on. There are plenty of alternatives so if you have suggestions for improvement, please comment!

Project Tasks

With any project it’s inevitable that you’ll begin writing scripts for maintenance. I decided to go with Grunt because it keeps me in the realm of JavaScript (even though the developer needs NodeJS installed). Plus there are lots of contributed Grunt tasks to choose from. For example, to minify my library at lib/myscript.js all I had to do was install uglify-js, grunt-contrib-uglify, and add a Gruntfile.js with this:

Now I can type this to minify the code:

grunt uglify

Running Unit Tests

All libraries (even JavaScript ones!) need automated tests otherwise you and future collaborators will find the library really hard to maintain. If you try to make a change to code without tests you have to think about every possible side effect or you have to manually test all side effects. Both of these things seem easy when you only have one or two functions but they become exponentially harder as your library grows.

I decided to try Karma to run tests in a real web browser, mocha for my test cases, and chai for assertions. There are a few alternatives but Karma seemed like the best choice for a runner. (I found out about Yeti after I settled on Karma; I haven’t tried it.)

Karma lets you run tests from your console against one or more web browsers in parallel. It’s exactly what I wish existed back when I wrote jstestnet! Karma is really fast and seems to work pretty well. Speed is essential in testing.

To get started, you can type karma init karma.conf.js to create a config file and you can hook it up to Grunt with grunt-karma in a Gruntfile like this:

This fit well with my project since I was already using Grunt. I can now type:

grunt test

Karma will open all target web browsers (per config file), run my test suite, report results, and shut down the web browsers.

The Karma output is pretty minimal. It looks something like this in a shell:

Screen Shot 2014-03-19 at 2.56.49 PM

For development I also added this command:

With that I can type:

grunt karma:dev

This opens all target web browsers, runs the test suite, and keeps the web browsers running. As I edit files, it re-runs the tests. This seems to work okay but occasionally I’ve seen some timeout errors that go away if I restart the browsers. I’m still looking into that.

Writing Tests

Mocha is a testing library that works in NodeJS and also in web browsers. It uses the BDD (behavioral driven development) style of specifying an object or function and declaring how it should behave. Here is an example from my library that covers some error handling:

As you can see it’s pretty easy to read the code and see what is being tested. Both Mocha and Jasmine have Karma adapters and there are probably other adapters too if you want to use another test library.

Testing With Mock Objects

A common pattern in testing is to mock out objects that are used by your system but that you don’t need to test. Sinon is designed exactly for this and especially for testing in the web browser. My library has a thin API layer around XMLHttpRequest but I wanted to mock that out while testing the API layer. Here is an example of making sure it gets an error callback for 500 responses:

This is nice because I can run my tests without the code touching a real API server. I’m using Sinon’s Fake Server here.

It’s easy to go overboard with mocks once you discover their power. My words of caution is that anytime you mock out an object you are deciding not to test something. Make sure that’s the right decision. Make sure it gives you real benefits like a speedup or something. It’s usually a good idea to mock out HTTP connections since otherwise your tests depend on the Internet.

Continuous Integration

Testing is most effective when you run your tests after every code commit. There is a free service for open source projects called TravisCI which supports running browser tests with Karma just fine. You can run all your tests in a headless browser like PhantomJS with a task like this:

Add the grunt command to a .travis.yml file to hook it up:

However, my specific library will benefit most from running its tests in a real Firefox and TravisCI supports web browsers just fine so, hey, why not. All I had to do was add this to my yaml file:

Firefox is pre-installed on TravisCI so I didn’t even need to declare it.

Checking For Syntax Errors

JavaScript is a quirky language (to put it nicely) so I always make sure to run the code through something like JS Hint to catch undeclared variables and other problems. You can add syntax checking easily to grunt like this using grunt-contrib-jshint:

This lets me type grunt jshint to check for syntax errors in all lib and test files as well as in my Gruntfile.js. I use a .jshintrc file to set common options that I like.

My continuous integration script actually runs JS Hint before running unit tests. I chain all these together like this at the bottom of my Gruntfile:

grunt.registerTask('test', ['jshint', 'karma:run']);

Now when I run grunt test the jshint and karma:run tasks are executed.

Documentation

All packages need documentation! I like to start with realistic scenarios to illustrate usage of the library. After that I make sure to fully document all public APIs. I tend to just put docs in a Github README and then switch to Sphinx and Read The Docs later when the docs are too big for a single README. Sphinx doesn’t support JavaScript as well as Python but it’s still probably the best tool out there for managing interlinked docs.

Packaging

To make a library useful to others you may wish to package it up. To be honest I haven’t done this to my library yet but I was considering bower and/or NodeJS packaging.

That’s It

Since I left out a lot of details, you may want to check the actual code for working examples.