Tracking deployments in Graphite

Andy McKay

1

We use Graphite in Web Development to track metrics on our sites. For the Marketplace and Addons team, it has been a real eye opener into the state of our sites.

One thing that I’ve wanted to add is information on when deployments occur as these are often critical periods that can trigger changes in the graphs. Sure enough, this has all been done before and covered in a blog post. Following on from that blog post, here’s how I added this for the Marketplace.

First, I added a management command to django-statsd that allows you to easily send a ping to Graphite for a specific key. The value of the key is set to be a UNIX time stamp. Once that was in place I altered our deployment scripts so that on each update they send a ping with the key “update”:

python2.6 manage.py statsd_ping --key=update

Then each time a server is deployed that key gets added to Graphite.

Next, I altered our graphs so that they show this ping as a vertical line. That’s using the command drawAsInfinite, for example:

drawAsInfinite(stats.timers.addons-dev.update.count)

Our development server updates on every commit. In other words, it continually deploys all the time. This is the graph for the development server, and you can see the large number of small commits we had this afternoon (each commit is indicated by a pink line):

A more realistic view is our production server, where you can see our Thursday afternoon push in context (each push is a pink line):

Adding this data is crucial to helping the Marketplace get closer to continuous deployment in the coming months.

One response

  1. Wil Clouser wrote on ::

    This is awesome. Nice work.