Bedrock, the code behind www.mozilla.org, is a very large Django project. It is mostly large due to the volume and diversity of independent pages it serves. These pages come with a surprising amount of static media (css, js, images, fonts, etc.). So, any system that we use to deal with said media should be efficient in order to keep our development servers fast.

We like django-pipeline for managing our static media in production. It does a great job of bundling, minifying, and compressing our css and js. When using it in a development environment however, it does not scale well. The issue is that it does not watch for changes to your files, so all it can do is copy them all from their source to the static directory on every page load. For a reasonable number of files this is probably fine, but as I said ours is not that. This is exasperated in slow I/O environments like Docker on a non-linux system (like OSX). We’ve not been able to setup an acceptable Docker-based local dev environment yet because it can literally take several minutes to render the home page.

Due to all of the issues noted above we’ve been looking for other ways of handling static media. We’ve considered a few times to move to a completely nodejs based system that would be completely independent of the Django side, and may still do that some day, but the problem has always been scope and impact. Again because the project is so large, making sweeping changes that affect workflow and all static files can both take a lot of time and be very disruptive when they land. So for a long time we figured we were stuck. But recently a conversation started in IRC about being able to just disable django-pipeline’s copying of files. If we could do that we could use gulp-watch to much more quickly and efficiently manage these files while being edited and still get the benefits of django-pipline for production. It turned out that someone else already had this idea and mostly we just needed to upgrade django-pipeline.

After that it was a simple matter of adding a task to our Gulpfile:

gulp.task('media:watch', function () {
    return gulp.src('./media/**/*')
        .pipe(watch('./media/**/*', {
            'verbose': true
        }))
        .pipe(gulp.dest('./static'));
});

But it was still a bit odd now having to have two shells open, one for the gulp task and another for the Django dev server. So we did a little more gulp magic and now have a single command to start up both the file watching and the Django server that combines the output of both in a single terminal.

gulp.task('serve:backend', function () {
    var devServerPort = process.env.PORT || 8000;
    process.env.PYTHONUNBUFFERED = 1;
    process.env.PYTHONDONTWRITEBITECODE = 1;
    spawn('python', ['manage.py', 'runserver', '0.0.0.0:' + devServerPort], {
        stdio: 'inherit'
    });
});

gulp.task('default', function() {
    gulp.start('serve:backend');
    gulp.start('media:watch');
});

You can see the full gulpfile.js in our repo on Github if you’d like. It’s a simple but very effective change and has made a very noticeable improvement in the speed of the local development server. And now we hope that we can finally complete and recommend Docker as the default setup for local development on bedrock.