I talked recently at DjangoCon about how to build better Python packages. As engineers we all use each other’s open source libraries everyday so it’s nice when 1) they work, 2) they’re well documented, and 3) they’re well tested.
This is a brief summary of some tools and patterns I settled on. There are plenty of alternatives so if you have suggestions for improvement, please comment!
lib/myscript.js all I had to do was install uglify-js, grunt-contrib-uglify, and add a Gruntfile.js with this:
Now I can type this to minify the code:
Running Unit Tests
I decided to try Karma to run tests in a real web browser, mocha for my test cases, and chai for assertions. There are a few alternatives but Karma seemed like the best choice for a runner. (I found out about Yeti after I settled on Karma; I haven’t tried it.)
Karma lets you run tests from your console against one or more web browsers in parallel. It’s exactly what I wish existed back when I wrote jstestnet! Karma is really fast and seems to work pretty well. Speed is essential in testing.
To get started, you can type karma init karma.conf.js to create a config file and you can hook it up to Grunt with grunt-karma in a Gruntfile like this:
This fit well with my project since I was already using Grunt. I can now type:
Karma will open all target web browsers (per config file), run my test suite, report results, and shut down the web browsers.
The Karma output is pretty minimal. It looks something like this in a shell:
For development I also added this command:
With that I can type:
This opens all target web browsers, runs the test suite, and keeps the web browsers running. As I edit files, it re-runs the tests. This seems to work okay but occasionally I’ve seen some timeout errors that go away if I restart the browsers. I’m still looking into that.
Mocha is a testing library that works in NodeJS and also in web browsers. It uses the BDD (behavioral driven development) style of specifying an object or function and declaring how it should behave. Here is an example from my library that covers some error handling:
As you can see it’s pretty easy to read the code and see what is being tested. Both Mocha and Jasmine have Karma adapters and there are probably other adapters too if you want to use another test library.
Testing With Mock Objects
A common pattern in testing is to mock out objects that are used by your system but that you don’t need to test. Sinon is designed exactly for this and especially for testing in the web browser. My library has a thin API layer around XMLHttpRequest but I wanted to mock that out while testing the API layer. Here is an example of making sure it gets an error callback for 500 responses:
This is nice because I can run my tests without the code touching a real API server. I’m using Sinon’s Fake Server here.
It’s easy to go overboard with mocks once you discover their power. My words of caution is that anytime you mock out an object you are deciding not to test something. Make sure that’s the right decision. Make sure it gives you real benefits like a speedup or something. It’s usually a good idea to mock out HTTP connections since otherwise your tests depend on the Internet.
Testing is most effective when you run your tests after every code commit. There is a free service for open source projects called TravisCI which supports running browser tests with Karma just fine. You can run all your tests in a headless browser like PhantomJS with a task like this:
Add the grunt command to a
.travis.yml file to hook it up:
However, my specific library will benefit most from running its tests in a real Firefox and TravisCI supports web browsers just fine so, hey, why not. All I had to do was add this to my yaml file:
Firefox is pre-installed on TravisCI so I didn’t even need to declare it.
Checking For Syntax Errors
This lets me type
grunt jshint to check for syntax errors in all lib and test files as well as in my Gruntfile.js. I use a .jshintrc file to set common options that I like.
My continuous integration script actually runs JS Hint before running unit tests. I chain all these together like this at the bottom of my Gruntfile:
grunt.registerTask('test', ['jshint', 'karma:run']);
Now when I run
grunt test the jshint and karma:run tasks are executed.
Since I left out a lot of details, you may want to check the actual code for working examples.