Bleg for a new machine: outcome

Recently I blegged (here and here) for help in designing a new machine.  My goals:  fast browser and JS shell builds, quietness, and a setup that wasn’t too complicated.  I now have the new machine and have done some comparisons to the old machine.

New vs old

The most important components of the new machine are:  an Intel i7-4770 CPU (I’m using the integrated graphics), 32 GiB of RAM, a 512GB Samsung 840 Pro SSD hard disk, and a Fractal Design Define R4 case.

In comparison, the equivalent components in the old machine were: an Intel i7-2600 CPU, 16 GiB of RAM, a magnetic hard disk, and an Antec Sonata III 500 case.

A basic comparison

The new machine is definitely faster.  Compile times are about 1.5x faster;  I can do a debug browser build with clang in under 13 minutes, and one with GCC in under 17 minutes.  (I hadn’t realized that clang was so much faster than GCC.)

Furthermore, disk-intensive operations are massively faster.  Just as importantly, disk-intensive operations vary in speed much less.  With a magnetic disk, if you’re doing something where the data is already in the disk cache, it’ll be pretty fast;  otherwise it’ll be horribly slow.  The SSD doesn’t suffer that dichotomy.

Finally, the new case, while not silent, is certainly quieter… maybe half as loud as the old one.  It’s also bigger than I expected — it’s 1–2 inches bigger in every dimension than the old one. There must be a lot of empty space inside.  And although it has a pleasingly minimalist aesthetic — it’s about as plain a black box as you could imagine — it does have an obnoxiously bright, blue power indicator light at the top of the front panel, which I quickly covered with a small strip of black electrical tape.

A detailed performance comparison

Building and testing

All builds are 64-bit debug builds.  I used clang 3.2-1~exp9ubuntu1 and gcc-4.7.real (Ubuntu/Linaro 4.7.3-1ubuntu1) for compilation. I measured each operation only once, and the old machine in particular would vary in its times due to the magnetic disk.  So don’t treat individual measurements as gospel.  In all cases I give the old machine’s time first.

  • Browser clobber build (clang): 19.7 minutes vs 12.7 minutes (1.56x faster).  I didn’t measure a GCC brower build on the old machine, but on the new machine it was 16.8 minutes (1.32x slower than clang).
  • Browser no-change build (clang): 48 seconds vs 31 seconds (1.55x faster).
  • Browser clobber build, with ccache, with an empty cache (clang): 23.3 minutes vs 14.8 minutes (1.57x faster).  These are 1.18x slower and 1.17x slower than the corresponding non-ccache builds.
  • Browser clobber build, with ccache, with a full cache (clang): 6.2 minutes vs 2.6 minutes (2.4x faster).  These are 3.18x faster and 4.89x faster than the corresponding non-ccache builds.  Here the effect of the SSD becomes clear — the new machine gets a much bigger benefit from ccache.
  • Two concurrent browser builds (clang): 45.9 & 45.4 minutes vs 22.5 & 22.5 minutes (2.03x faster).  Interestingly, the amortized per-build time on the old machine (22.9 minutes) was 1.16x slower than a single build, but the amortized per-build time on the new machine (11.3 minutes) was 1.12x faster than a single build.  The new machine, despite having the same number of cores, clearly provides more parallelism, and a single browser build doesn’t take full advantage of that parallelism.
  • JS shell everything-but-ICU build (clang): 59 seconds vs 42 seconds (1.4x faster).  It’s worth noting that JS shell builds spend a higher proportion of their time doing C++ compilation than browser builds.
  • JS shell everything-but-ICU build (GCC): 130 seconds vs 81 seconds (1.60x faster).  These are 2.20x slower and 1.93x slower than the corresponding clang builds!
  • JS jit tests (compiled with clang): 179 seconds vs 137 seconds (1.31x faster).  These tests are much more CPU-bound and less disk-bound than compilation, so the smaller speed up isn’t surprising.
  • SunSpider: 156 ms vs 127 ms (1.23x faster).  Again, CPU is the main factor.

Next, here are the times for some disk-intensive operations.  The results here, especially for the old machine, could be highly variable.

  • Delete a build directory: 10.5 seconds vs 1.4 seconds (7.5x faster).
  • Do a local clone of mozilla-inbound: 7.7 minutes vs 10 seconds (46x faster).
  • Recursive grep of .cpp/.h/.idl files in a repository, first time: 53.2 seconds vs 0.8 seconds (67x faster).
  • The same operation, immediately again: 0.2 seconds vs 0.2 seconds (same speed).

Those last two comparisons really drive home the impact of the SSD, and the reduction in variability it provides. It’s hard to describe how pleasing this is.  On the old machine I always knew when libxul.so was linking, because my whole machine would grind to a halt and trivial operations like saving a file in vim would take multiple seconds.  I don’t have that any more!

And this is relevant to ccache, too.  I tried ccache again recently on my old machine, and while it did speed up compilations somewhat, the extra load on the disk noticeably affected everything else — I had even more of those unpredictable pauses when doing anything other than building.  This was annoying enough that I disabled it.  But ccache should be much more attractive on the new machine.  I will try it again soon, once I’ve had the new machine long enough that I will be well-attuned to its performance.

Conclusion

The CPU is a decent improvement over the old one.  It accounts for roughly half the improvement in build times.

The SSD is a fantastic improvement over the old one.  It too accounts for roughly half the improvement in build times, but makes disk-intensive operations much faster.  It’s performance is also much less variable and thus more predictable.

clang is up to 2x faster than GCC!  This surprised me greatly.  I’d be interested to hear if others have seen such a large difference.

Thanks again to everybody who helped me design the new machine.  It’s been well worth the effort!

17 Responses to Bleg for a new machine: outcome

  1. Hi Nic, what about the virtual machine? Which one will you use and how is the performance?

    • Nicholas Nethercote

      I haven’t done that yet. Apparently I can get a VMWare license from Mozilla so I might do that. The performance of the VM won’t be a high priority, because I use Windows so infrequently.

  2. So, what about the power consumption of the new vs old machines?
    Does the new machine require less energy (fewer watt-seconds (Joules))
    to build the browser?

  3. I’m about to buy the same case and CPU, and I want to ask you, are you using the intel stock fan? Is it noisy? Have you tried cheking the temperature during load?
    Another thing: I know the led is very bright, some people connect it to the hdd instead of power so it blinks, also because the case have only one light instead of two.
    If you have any other impression, good or bad, I would be pleased to know

    • Nicholas Nethercote

      It’s pretty quiet; certainly not silent, but quieter than my old case.

      I’m probably using the Intel stock fan. I don’t know because I didn’t assemble it myself, but I didn’t order any special fans. There’s just one fan at the back, but there is space for two fans on top which I’m not using. Presumably it would be noisier with those additional fans.

      I haven’t checked the temperature.

      Overall I’m pretty happy. The only other thing I’d add is that it feels solid and well made, partly because it’s quite heavy (12.3kg IIRC).

      • Nicholas Nethercote

        Oh, there’s space for a fan on the left side, too.

      • If you’re curious enough to pop the case open; telling if you’ve got the Intel cooler or an aftermarket one is easy. Intel’s coolers have a ~80mm fan that blows down onto the motherboard; most aftermarket coolers are towers with a 92/120mm fan blowing parallel to the mobo surface (typically into the rear exhaust fan).

  4. The size creep you saw with the Fractal Define is more or less par for the course for the last few years. The extra width is to give room for cable routing behind the motherboard. Top mounted fans are now a standard feature; that’s an inch for the fans, 2 or 3 if the case is also making room for a water cooling radiator (and all in one kits are making that more common). Front to back doors with side vents and a chamber between themselves and the fans do a lot to suppress noise without impeding cooling performance. Depending on what your old case was you might also be seeing the tail end of very high end GPUs growing to ~11.5″ long (an inch past the end of the mobo) and needing more space to fit as a result.

    The main countervailing factor is that mATX and mITX boards are much closer to the capabilities of their larger brethren than they used to be allowing a lot of people to downsize their system by getting a smaller board.

  5. I just got a machine almost identical to yours, the only difference being that I got a i7 4770K with water cooling. I overclocked the CPU in BIOS to 4GHz (which was super easy to do), and using gcc 4.7 and ld, it just did a clobber build on Linux in 16:19.

    Windows clobber builds take 33:25 on this machine, which is only 20% better than the build times in Windows on my Lenovo W530 laptop, which is disappointing.

    I tried overclocking to 4.5GHz, but under windows the build time was still around 33 minutes and the cooling started to make noise. When I clocked it to 4.7GHz the machine started crashing, so I settled on clocking it at 4GHz in the end.

  6. clang is up to 2x faster than GCC!
    Do compiled binaries have performance in browsers benchmarks? Clang often blamed for producing slower binaries

    • Sorry, missed word “same” in first sentence

    • Even if they are slower, for what NJN is doing (testing to make sure his changes don’t make anything break), the compile time savings are almost certainly worth it. Unless you’re testing performance enhancements, only your automated testing and release build servers need to be configured for maximum app performance at the cost of slower compile times.

  7. Congratulations for your new setup, he is another FF dev guy, who tested VM vs Virtualbox.. You should read his blogpost:
    http://boomswaggerboom.wordpress.com/2013/09/13/informal-test-building-firefox-with-vmware-vs-virtualbox/