While the Firefox Marketplace is being built, we are starting to make some fundamental changes in the way the Marketplace is constructed behind the scenes. In the beginning there was one big Django site that served all the requests back to the users.

Over the course of the last few months that has changed to being a smaller number of services that provide APIs to each other. We’ve got separate services for payments (and this), statistics (and this) and a new front end client and plans for more. The main communication mechanism between them is going to be REST APIs.

For REST APIs in Django we are currently using Tastypie, which does a pretty good job of doing a lot of things you’d need. There are a few frustrations with Tastypie and going forward I’m extremely tempted by Cornice, which we currently use for statistics.

When you ask people about consuming REST APIs in Python, lots of people tell me “we just use requests“. Requests is a great library for making HTTP calls, but when you are developing against a REST API having all the overhead of coping with HTTP is a bit much. Coping with client errors versus HTTP errors, understanding the error responses, scaling and failover and generally coping with an imperfect world.

So we took slumber and wrapped that in our our library called curling. Curling is a wrapper that makes some assumptions about the server and the client. It provides us one entry point for all our REST APIs and a place to have consistency on the client side. Below I get a timestamp on a transaction using requests.

However in the curling example it will check that one and only one result is returned (and raise meaningful errors if it didn’t) and correctly raise a meaningful error based on the HTTP response.

The result is code that is easier to write and read, but it is still relatively close to the metal of just being a REST API without being as simple as just HTTP requests or as complicated as a Web Services and SOAP stack.

As an extra bonus curling has a command line API that fills some HTTP headers out for you, syntax highlights output and the like. As a bonus, if you are calling a Django server in debug mode, rather than spew lines of HTML to the screen – it will write the HTML to a file and open a browser to the file.

These APIs are open and growing rapidly. You can find documentation on them on each project, but the main one to follow is in zamboni.