{"id":1994,"date":"2011-10-25T15:15:29","date_gmt":"2011-10-25T23:15:29","guid":{"rendered":"http:\/\/blog.mozilla.org\/webdev\/?p=1994"},"modified":"2011-10-28T09:53:22","modified_gmt":"2011-10-28T17:53:22","slug":"go-speed-racer","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/","title":{"rendered":"From 80 Seconds to 6: Optimizing Our Asset Compression"},"content":{"rendered":"<p>Before pushing our CSS and JavaScript assets to our CDN, we run them through <a href=\"https:\/\/github.com\/jsocol\/jingo-minify\">jingo-minify<\/a> to concat and minify the files, as well as cache bust them and any resources (such as images) contained inside them.\u00a0 Turns out, this was by far the slowest part of our push process \u2014 it took\u00a0 <strong>between<\/strong> <strong>80 and 160 seconds <\/strong>for addons.mozilla.org (AMO) assets.\u00a0 It wasn&#8217;t a huge priority, since most of the time this happens in the background and nobody really notices.\u00a0 However, I wanted to see how fast I could get it.<!--more--><\/p>\n<p><em>Note: All times are based on AMO being compressed on a MacBook.\u00a0 On our build server, those 80 seconds took about 160 seconds.\u00a0 Most times are averages and\/or estimates, and are based on an initial 80 seconds.<br \/>\n<\/em><\/p>\n<h3>UglifyJS and clean-css (-43s)<\/h3>\n<p>This was the low hanging fruit.\u00a0 YUICompressor is slow.\u00a0 We managed to cut off over half our original time by switching to <a href=\"https:\/\/github.com\/mishoo\/UglifyJS\">UglifyJS<\/a> for JavaScript and <a href=\"https:\/\/github.com\/GoalSmashers\/clean-css\">clean-css<\/a> for CSS, both of which are written in Node.\u00a0 This switch means less errors (we were getting a lot with YUI), smaller files and much faster build times \u2014 and none of us are sad to see our Java dependency disappear.\u00a0 (I benchmarked all our options before we chose them.\u00a0 If you&#8217;re interested, you can <a href=\"https:\/\/github.com\/gkoberger\/compare-minifiers\">run them yourself<\/a>.)<\/p>\n<p>Unlike YUICompressor, UglifyJS builds an abstract syntax tree, compresses the AST, then converts the whole thing back to (minified) JavaScript.\u00a0 So far, this has meant that UglifyJS has been much more resilient against our sometimes-misformed JavaScript. Missed semicolons no longer take down the whole site&#8217;s JavaScript. The build times are about the same by my benchmarks, but the results are far superior. [<a href=\"http:\/\/pastie.org\/2564736\">JS Benchmark results<\/a>]<\/p>\n<p>Clean-css works much like YUI (lots of regex), which is a bit unfortunate.\u00a0 However, it does so much faster \u2014 10x as fast, to be exact.\u00a0 [<a href=\"http:\/\/pastie.org\/2571130\">CSS Benchmark results<\/a>]<\/p>\n<p><strong>Original Time:<\/strong> 80s<strong><br \/>\nNew Time: <\/strong>37s<br \/>\n<strong>Difference:<\/strong> -54%<\/p>\n<h3>Saving File Hashes (-2s)<\/h3>\n<p>We were fetching the hash (originally the git commit id, now hash of the actual file) for the same files fairly often.\u00a0 By sticking the already-computed hashes into a dict, we were able to shave off a small but significant amount of time.<\/p>\n<p><strong>Original Time:<\/strong> 37s<strong><br \/>\nNew Time: <\/strong>35s<br \/>\n<strong>Difference:<\/strong> -11%<\/p>\n<h3>Use File Hashes For Image Cache Busting (-12s)<\/h3>\n<p>When I first wrote cache busting for individual images in our CSS files, I used the image&#8217;s most recent git commit as the cache-busting hash.\u00a0 As it turns out, your script gets really slow when you&#8217;re looking up the history for dozens of images using <a href=\"http:\/\/gitorious.org\/git-python\">GitPython<\/a>.\u00a0 So, I switched to using a hash of the file.\u00a0 The end result is the same \u2014 the hash is only updated if the image is changed.\u00a0 However, using file hashes instead of git commits shaved a good chunk of time off our build process.<\/p>\n<p><strong>Original Time:<\/strong> 35s<strong><br \/>\nNew Time: <\/strong>23s<br \/>\n<strong>Difference:<\/strong> -34%<\/p>\n<h3>Only Minify When Necessary (up to -17s)<\/h3>\n<p>I got down to 23 seconds, and was stuck.\u00a0 The biggest time sink in the process was the minifying of files, which I couldn&#8217;t do much about without touching the UglifyJS or clean-css code.\u00a0 Even if I wanted to, I doubted I could do much to speed either of them up.\u00a0 After thinking about it for a bit, I realized that there was no reason for us to be minifying every file each time.\u00a0 We only needed to minify the files that had actually changed.\u00a0 So, jingo-minify now writes the concatted file out to a temporary location, and compares it to the preexisting concatted file.\u00a0 If they&#8217;re the same file, we have no need to re-minify.<\/p>\n<p>Since we push to preview every time anyone commits and most commits don&#8217;t modify more than one or two JS or CSS files (if any), we can save between 15 and 17 seconds each time.<\/p>\n<p><strong>Original Time:<\/strong> 23s<strong><br \/>\nNew Time: <\/strong>6s<br \/>\n<strong>Difference:<\/strong> -74%<\/p>\n<h3>Overall Results<\/h3>\n<p>So, that&#8217;s how I managed to speed up our asset compression by over an order of magnitude \u2014<strong> from 80 seconds down to about 6<\/strong>.\u00a0 <em>(Note: Time is based on the script on a MacBook.\u00a0 On the slower build server, it&#8217;s closer to 160 seconds down to 9.)<\/em>\u00a0 Hopefully this means much smoother pushes for AMO, SUMO and anyone else using jingo-minify to compress their assets.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Before pushing our CSS and JavaScript assets to our CDN, we run them through jingo-minify to concat and minify the files, as well as cache bust them and any resources (such as images) contained inside them.\u00a0 Turns out, this was &hellip; <a class=\"go\" href=\"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/\">Continue reading<\/a><\/p>\n","protected":false},"author":269,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[288],"tags":[],"coauthors":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>From 80 Seconds to 6: Optimizing Our Asset Compression - Mozilla Web Development<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Gregory Koberger\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/\",\"url\":\"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/\",\"name\":\"From 80 Seconds to 6: Optimizing Our Asset Compression - Mozilla Web Development\",\"isPartOf\":{\"@id\":\"https:\/\/blog.mozilla.org\/webdev\/#website\"},\"datePublished\":\"2011-10-25T23:15:29+00:00\",\"dateModified\":\"2011-10-28T17:53:22+00:00\",\"author\":{\"@id\":\"https:\/\/blog.mozilla.org\/webdev\/#\/schema\/person\/8d3fbcbd19a1fc80e76a4ad481e517e2\"},\"breadcrumb\":{\"@id\":\"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/blog.mozilla.org\/webdev\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"From 80 Seconds to 6: Optimizing Our Asset Compression\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.mozilla.org\/webdev\/#website\",\"url\":\"https:\/\/blog.mozilla.org\/webdev\/\",\"name\":\"Mozilla Web Development\",\"description\":\"For make benefit of glorious tubes\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.mozilla.org\/webdev\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.mozilla.org\/webdev\/#\/schema\/person\/8d3fbcbd19a1fc80e76a4ad481e517e2\",\"name\":\"Gregory Koberger\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/webdev\/#\/schema\/person\/image\/228dcc5038736893bcdf50cba284a617\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/0c98fd738dc3454f5a1649fdd25a8b87?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/0c98fd738dc3454f5a1649fdd25a8b87?s=96&d=mm&r=g\",\"caption\":\"Gregory Koberger\"},\"description\":\"Gregory Koberger is a front- and back-end developer for addons.mozilla.org. You can view his portfolio at gkoberger.net, or follow him at @gkoberger.\",\"sameAs\":[\"http:\/\/gkoberger.net\"],\"url\":\"https:\/\/blog.mozilla.org\/webdev\/author\/gkobergermozilla-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"From 80 Seconds to 6: Optimizing Our Asset Compression - Mozilla Web Development","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/","twitter_misc":{"Written by":"Gregory Koberger","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/","url":"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/","name":"From 80 Seconds to 6: Optimizing Our Asset Compression - Mozilla Web Development","isPartOf":{"@id":"https:\/\/blog.mozilla.org\/webdev\/#website"},"datePublished":"2011-10-25T23:15:29+00:00","dateModified":"2011-10-28T17:53:22+00:00","author":{"@id":"https:\/\/blog.mozilla.org\/webdev\/#\/schema\/person\/8d3fbcbd19a1fc80e76a4ad481e517e2"},"breadcrumb":{"@id":"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/blog.mozilla.org\/webdev\/2011\/10\/25\/go-speed-racer\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blog.mozilla.org\/webdev\/"},{"@type":"ListItem","position":2,"name":"From 80 Seconds to 6: Optimizing Our Asset Compression"}]},{"@type":"WebSite","@id":"https:\/\/blog.mozilla.org\/webdev\/#website","url":"https:\/\/blog.mozilla.org\/webdev\/","name":"Mozilla Web Development","description":"For make benefit of glorious tubes","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.mozilla.org\/webdev\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/blog.mozilla.org\/webdev\/#\/schema\/person\/8d3fbcbd19a1fc80e76a4ad481e517e2","name":"Gregory Koberger","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/webdev\/#\/schema\/person\/image\/228dcc5038736893bcdf50cba284a617","url":"https:\/\/secure.gravatar.com\/avatar\/0c98fd738dc3454f5a1649fdd25a8b87?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/0c98fd738dc3454f5a1649fdd25a8b87?s=96&d=mm&r=g","caption":"Gregory Koberger"},"description":"Gregory Koberger is a front- and back-end developer for addons.mozilla.org. You can view his portfolio at gkoberger.net, or follow him at @gkoberger.","sameAs":["http:\/\/gkoberger.net"],"url":"https:\/\/blog.mozilla.org\/webdev\/author\/gkobergermozilla-com\/"}]}},"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/webdev\/wp-json\/wp\/v2\/posts\/1994"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/webdev\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/webdev\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/webdev\/wp-json\/wp\/v2\/users\/269"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/webdev\/wp-json\/wp\/v2\/comments?post=1994"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/webdev\/wp-json\/wp\/v2\/posts\/1994\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/webdev\/wp-json\/wp\/v2\/media?parent=1994"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/webdev\/wp-json\/wp\/v2\/categories?post=1994"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/webdev\/wp-json\/wp\/v2\/tags?post=1994"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/webdev\/wp-json\/wp\/v2\/coauthors?post=1994"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}