{"id":3215,"date":"2019-10-10T10:34:50","date_gmt":"2019-10-09T23:34:50","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=3215"},"modified":"2019-10-10T10:34:50","modified_gmt":"2019-10-09T23:34:50","slug":"visualizing-rust-compilation","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2019\/10\/10\/visualizing-rust-compilation\/","title":{"rendered":"Visualizing Rust compilation"},"content":{"rendered":"<p>Speeding up the Rust compiler isn&#8217;t the only way to make a Rust project build faster. Changing the crate structure of a project can also make a big difference. The good news here is that <a href=\"https:\/\/github.com\/ehuss\">Eric Huss<\/a> has implemented an <a href=\"https:\/\/internals.rust-lang.org\/t\/exploring-crate-graph-build-times-with-cargo-build-ztimings\/10975\">amazing tool<\/a> for visualizing Rust compilation, which can be used to identify inefficient crate structures in Rust projects.<\/p>\n<p>The tool extremely easy to use. First, update to the latest Nightly:<\/p>\n<pre>rustup update nightly<\/pre>\n<p>Then just add <code>-Ztimings<\/code> to your build command, e.g.:<\/p>\n<pre>cargo +nightly build -Ztimings<\/pre>\n<p>At the end of the build it will print the name of an HTML file containing the data. Here&#8217;s part of the visualization for the Rust compiler itself:<\/p>\n<p><a href=\"http:\/\/blog.mozilla.org\/nnethercote\/?attachment_id=3214\" rel=\"attachment wp-att-3214\"><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-3214 size-full\" src=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2019\/10\/a.jpeg\" alt=\"Screenshot of -Ztimings output when compiling the Rust compiler\" width=\"2280\" height=\"1536\" srcset=\"https:\/\/blog.mozilla.org\/nnethercote\/files\/2019\/10\/a.jpeg 2280w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2019\/10\/a-252x170.jpeg 252w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2019\/10\/a-768x517.jpeg 768w, https:\/\/blog.mozilla.org\/nnethercote\/files\/2019\/10\/a-600x404.jpeg 600w\" sizes=\"(max-width: 2280px) 100vw, 2280px\" \/><\/a><\/p>\n<p>Full data is available <a href=\"http:\/\/gistpreview.github.io\/?74d799739504232991c49607d5ce748a\">here<\/a>. (I recommend moving the &#8220;Scale&#8221; slider to 7 or 8 so that horizontal scrolling isn&#8217;t necessary.)<\/p>\n<p>Two things leap out from this visualization.<\/p>\n<ul>\n<li>The rustc crate takes about twice as long as any other crate to compile. It is the &#8220;long pole&#8221; of the build, and its presence serializes the build significantly. Breaking it up could improve compilation time quite a bit. I filed <a href=\"https:\/\/github.com\/rust-lang\/rust\/issues\/65031\">#65031<\/a> about this.<\/li>\n<li>Pipelined compilation (released in <a href=\"https:\/\/blog.rust-lang.org\/2019\/09\/26\/Rust-1.38.0.html\">Rust 1.38<\/a>) is a huge win for the compiler itself. Pipelining allows a dependent crate to start building as soon as metadata is produced. In the visualization, this corresponds to point where the bar for a graph changes colour from light blue to purple. Imagine if the rustc crate had to finish before all the crates below it could even start! It used to take about 45 minutes for an optimized stage 2 build on my fast Linux desktop machine; thanks to pipelining it now takes about 26 minutes.<\/li>\n<\/ul>\n<p>I also filed <a href=\"https:\/\/github.com\/rust-lang\/rust\/issues\/65088\">#65088<\/a>\u00a0to add <code>-Ztimings<\/code> support to the Rust compiler&#8217;s own build system. (Enabling the visualization isn&#8217;t as simple for the compiler as it is for most Rust projects. The compiler&#8217;s build system is complicated by the fact that it&#8217;s a bootstrapping compiler that has to be built multiple times.)<\/p>\n<p>We have already heard from multiple people that they used it fix inefficiencies in their crate structure, speeding up their builds significantly. Anyone who works on a sizeable Rust project should try out this tool.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Speeding up the Rust compiler isn&#8217;t the only way to make a Rust project build faster. Changing the crate structure of a project can also make a big difference. The good news here is that Eric Huss has implemented an amazing tool for visualizing Rust compilation, which can be used to identify inefficient crate structures [&hellip;]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[311,16179],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/3215"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=3215"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/3215\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=3215"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=3215"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=3215"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}