Developing with Vagrant, Puppet, and playdoh

Matthew Riley MacPherson

1

Mozilla is a large, highly visible, open source organization, with loads of contributors who commit code to, document, and localize our products. A big part of our success is our community, and our web apps are a big part of our product base.

Logistically, it’s very easy to commit code to our web apps. Most of them are hosted on GitHub: all you need to do is fork our code, commit your changes, and send us a pull request. We can review your contributions, comment on your code, discuss the changes, and (hopefully) merge in your awesome new code. Most of this can even be done entirely in a web browser!

This process is pretty straightforward, and it’s how we do all code reviews for both contributors and employees. I think we’re all really happy GitHub and its pull request system exist.

But even with this incredibly common and easy system for contributing code to Mozilla Webdev projects, many of our web apps — from smaller apps like Firefox Input to huge projects like Firefox Support — don’t have a large outside contributor community; most of the committers are Mozilla employees. And honestly, it’s not hard to tell why:

It’s bloody hard to get these apps up and running.

All the Code is Somewhere Else!

A lot of the code that powers Mozilla’s web apps are in libraries outside the main app’s repository. This is awesome: we reuse a lot of code and can keep things compartmentalized. We have our own Django app template called playdoh, powered by a bunch of libraries in funfactory. But sometimes getting all of this stuff setup is confusing and time-consuming.
Shut up and take my code!
The Firefox Input README details the steps to get the Input development server running. It includes installing virtualenv, pip, a set of python libraries (including ones with native C extensions), Sphinx, a MySQL database, running the Sphinx indexer… geez! I just want to make a CSS patch and test it — what gives?!

This setup pain doesn’t just grind contributors’ gears; employees often have to spend a day or so getting an app’s environment setup just so they can start working. This is a serious waste of everyone’s time; my time could be better spent coding than making sure my compiler arch flags are set properly or that MySQL is setup properly.

Dammit Jim: I’m coder, not a sysadmin!

The trick here is to get the computer to do as much of this menial and often fragile setup for us through automation/scripts.

Wait? Don’t we do that already? Isn’t that what README.markdown and requirements.txt are for?

C’mon, what is this, two thousand ten? It’s the freakin’ future, man. Instructions for humans are nowhere near as good as instructions for computers. Humans make mistakes, misunderstand, and are generally not as fast as computers when doing this kind of thing. We can tell the computer to do stuff on our behalf while we watch videos of adorable animals!

So in the spirit of doing less annoying work, dealing with less setup Hell, and giving more developers easy access to local development of playdoh apps, we’ve added Vagrant and Puppet support to¬† playdoh!

Um, What?

Yo dawg I heard you like developing so I developed a development VM.

Ah, OK, I’ll explain:

Vagrant is a simple command-line utility that boots up customized virtual machines you can access from your host machine. Puppet is a scripting system that allows you to instruct a machine to install certain libraries, start services, and run custom commands. With these two utilities, all it takes to get access to a fully ready-to-run playdoh app are two simple commands:

vagrant up
vagrant ssh

That’s way better than Input’s huge README.

Allow Me to Break it Down

The first command downloads a virtual machine image (right now we’re using Ubuntu Linux), boots it, then applies our Puppet provisioning scripts. Puppet installs Apache 2, MySQL, memcached, Python libraries, SQL migrations, and other custom jazz. It’s all done without any user input.

The second command gives you access to a machine with a folder pointing to a directory on your host machine, so you can develop in your host operating system (which could be Windows, Mac OS X, or Linux/UNIX) but run code on your guest VM. This whole process takes about as long as it does to brew a pot of coffee or listen to a Collective Soul song.

We’ve actually been using Vagrant and Puppet for a few months now on our recently-launched Mozillians directory app. Mozillians isn’t a standard playdoh app; it’s got both a MySQL and an LDAP backend, with some fairly custom libraries and setup requirements. That said, ednapiranha — a new employee at Mozilla — got the development environment up and running in under fifteen minutes without knowing much about playdoh or LDAP. That’s incredibly promising, and we’ve only been at this a few months.

Moving Forward

Ducati motorcycle on a long, open road
This is just the beginning. playdoh’s vagrant/puppet setup is pretty simple right now, but we’re planning on adding more to it. We hope to get to a point where more complex apps like Add-ons can run inside our VM. It would also be sweet to have a VM emulate our production environment so the behaviours we see locally are as close to our production servers as possible. And, of course, you can still run playdoh apps locally without using Vagrant.

But really, why do you wanna do all that work when you could be smiling at adorable kittens on YouTube?

Contributing

Wanna check this out? Head over to the playdoh repo on GitHub and check out the playdoh docs on Vagrant/Puppet. If you see a bug or have a cool idea for a feature in playdoh: send us a pull request!

It should be easier now than ever.

One response

  1. David Boswell wrote on ::

    This all sounds very cool. Since Mozilla is focused on the web and all it seems like getting more people involved with our webdev efforts would be a natural fit. Good to see efforts like this to make it easier for people to contribute.