MemShrink Performance Personal Rust

Farewell, Mozilla

Today is my last day working for Mozilla. I will soon be starting a new job with Apple.

I have worked on a lot of different things over my twelve years at Mozilla. Some numbers:

  • Three years as a contractor, and nine as an employee.
  • 4,441 commits to mozilla-central, 560 to rustc, 148 to rustc-perf, and smaller numbers to several other repositories.
  • 2,561 bugs filed in Bugzilla, 2,118 bugs assigned to me, 27,647 comments, 2,411 patches reviewed.
  • Three module peerages and one module ownership.
  • 277 blog posts.
  • Six managers and four managees, across three teams. (One of my managees later became my manager. Thankfully, it worked well!)
  • More trans-Pacific air miles than I want to count.

Two areas of work stand out for me.

  • I started the MemShrink project and for several years played the roles of tech lead, engineering project manager, engineer, and publicist. It changed Firefox’s memory consumption from its biggest technical weakness into a strength, and enabled the use of more processes in Electrolysis (for responsiveness) and Fission (for security).
  • My work on the Rust compiler, rustc-perf, and related profilers helped the compiler become roughly 2.5x faster over a three year period, and laid a foundation for ongoing future improvements.

I have a lot of memories, and the ones relating to these two projects are at the forefront. Thank you to everyone I’ve worked with. It’s been a good time.

As I understand it, this blog will stay up in read-only mode indefinitely. I will make a copy of all the posts and if it ever goes down I will rehost them at my personal site.

All the best to everyone.


On vacation for a month

I’m taking a month of vacation. Today is my last working day for March, and I will be back on April 30th. While I won’t be totally incommunicado, for the most part I won’t be reading email. While I’m gone, any management-type inquiries can be passed on to Naveed Ihsannullah.


A story about Brendan Eich

I attended a Mozilla work week a couple of years ago at Mozilla’s Mountain View office. There was a dinner event in San Francisco and, by chance, I ended up in Andreas Gal’s car, along with Brendan Eich and someone else (who, alas, I cannot remember now).

The destination was the California Academy of Sciences, a science museum in San Francisco, which was about a 45 minute drive away. Off we headed. Unfortunately, we headed off without closely checking where our destination was, and we somehow got the Academy of Sciences confused with the Exploratorium, another science museum in San Francisco. When we arrived and found it closed, we had to regroup.

Andreas confidently interrogated his car’s GPS unit and procured a new address that fortunately wasn’t too far away. Fifteen minutes later, we found ourselves in a residential area, outside a building that obviously wasn’t going to be hosting a dinner for several dozen MoCo employees.

Andreas again consulted his GPS unit for a new address. Unfortunately, this one
was on the far side of the city. Undeterred, we crawled through early-evening
traffic in the busiest parts of San Francisco — I’m pretty sure we actually
passed Union Square — to another address. Again, as soon as we laid eyes upon it, it clearly wasn’t the right destination.

It turns out there are several institutions in San Francisco with the words
“Academy” and “Science” or “Sciences” in their names, and we were doing a tour of all the wrong ones. On our fourth roll of the dice, Andreas found what ultimately was the correct address, and we crawled back to our final destination, which turned out — groan — to be not that far from the Exploratorium. We staggered in, two hours after we started, eliciting several comments of “what on earth took you guys so long?”

I remember being frustrated at the time — Andreas and Brendan were locals!
They should have known better. But now…

I’ve worked for Mozilla for over five years, but I visit California infrequently, and I’ve only had a chance to talk with Brendan in person a few times. The only
thing I remember from the conversation during the car trip is that at one point we were talking about the US economy and Brendan made a confident proclamation about the bond market — I can’t even remember what it was — that I wasn’t sure I agreed with but I wasn’t sure I could explain why I disagreed. It’s funny the details that stick.

This conversation was with Brendan the person — not Brendan the CTO, not
Brendan the inventor of JavaScript, not my boss’s boss’s boss, and not somebody who made a donation. Just Brendan, a person who knew a lot of stuff, had some interesting experiences and some strong opinions, and was good to chat to. It’s a small story, but it’s one I’ll remember.

Comments on this post are open, but be warned that I will delete without hesitation any comments that re-hash the CEO controversy of the past two weeks, or that I find rude or objectionable in any way. If you want to discuss the controversy, or be rude or objectionable, there are many other places on the web that you can do so.


On vacation until June 3rd

I’m about to go on vacation.  I won’t be back until June 3rd, and I will be checking email sporadically at best.

I look forward to a mountain of bugmail when I return.

Firefox Personal

Browser flavours

My daughter Keira turns four in a few months.  She’s been practising reading the letters on my Firefox hoodie.  On Saturday, we started with the front.

Keira: F – I – R – E – F – O – X

Me: What does that spell?

Keira: Firefox!

Then we moved to the back.

Keira: M – O – Z – I – L – L – A

Me: What does that spell?

Keira: <pause> Vanilla.

It’s clearly time to rename “add-ons” as “crush-ins”.

AdBlock Plus Firefox Google Chrome Personal

Converting a Chrome user to Firefox

[Update: before commenting, you should probably read this follow-up post that clarifies certain things about this post.]

On my recent vacation I was staying with a family member, let’s call her Karen (not her real name).  She was a Google Chrome user, and I managed to convert her to a satisfied Firefox user.  Here’s what I learnt along the way.


Bad things about the experience:

  • The third-party add-ons situation on Windows is awful.
  • We need a “import history from Chrome” feature.

Good things about the experience:

  • Mozilla’s non-profit nature is compelling, if you know about it.
  • AdBlock Plus is great.

The Initial Situation

Karen is a moderately sophisticated computer user. She knows what a browser is, but didn’t know how many there were, who made them, or any notable differences between them.

Her machine that is probably 2 or 3 years old, and runs Windows Vista.  She had used IE in the past (not sure which version) but didn’t like it, switched to Chrome at some point — she didn’t remember how or why — and found it to be much better.  She was running Chrome 14.0.835.202 (no, that’s not an IP address!) which was the latest stable version.

She also had Firefox 3.6.17 installed, but judging from the profile she hadn’t used it much — there was very little history.  She had the following Firefox add-ons installed:

  • Java Console 6.0.20 and 6.0.27.
  • The .NET Framework Assistant.
  • Some media player thing.
  • Some Norton “safe search” toolbar, and Symantec IPS, whatever that is.

(What is the Java Console?  What is the .NET Framework Assistant?  As far as I can tell they are (a) very common and (b) useless.)

I told her that I worked on Firefox and suggested that she try it and she was open to the idea.  I talked about the differences between Firefox and Chrome and some of my work on Firefox.  The thing that caught her attention most was that Mozilla is a non-profit organization.  She hadn’t known this and it appealed to her greatly — she said that browser speed and the non-profit nature were the two most important things to her.  She was also somewhat interested when I said that Firefox had an ad-blocking add-on.  At the end of the conversation, she agreed to let me install Firefox and make it the default browser.

Installing Firefox

I removed the existing Firefox profile manually — I wasn’t sure if this was necessary, but I definitely wanted a fresh profile — and then uninstalled Firefox through the Control Panel.  I then installed Firefox 7.0.1.  (BTW, I stayed with Karen for two weeks at this point, and I had deliberately waited until Firefox 7 was out before doing this because I knew it had much lower memory usage than Firefox 6.)

An unexpected thing was that the Firefox installer asked me to close all the other running programs;  it explained that this would mean that I wouldn’t have to reboot.  I’m used to running Firefox on Mac and Linux so I’m not used to this, but I’m familiar enough with Windows that I wasn’t totally surprised.  Still, it was annoying;  Karen had MS Word and some other programs open and I had to go ask her if I needed to save anything before closing them.  I realize this is Windows’ fault, not Firefox’s, but it was an obstacle.

Starting Firefox

When I started Firefox it asked me if I wanted to make it the default browser and I said yes.  (I explained to Karen how to switch the default browser back to Chrome if she was unhappy with Firefox.)

It also asked me if I wanted to import history/bookmarks/etc from IE.  But there was no equivalent for Chrome!  Karen had a ton of bookmarks in Chrome, but fortunately she said she only used a handful of them so I was able to copy them manually into Firefox’s bookmarks toolbar (which I had to make visible).  I’ve heard that someone is working on an “import from Chrome”  feature to Firefox but I don’t know what the status is.  We need it badly.

Once Firefox started, another unexpected thing was the state of the add-ons in the new profile.  The Symantec add-ons (including the ugly Norton toolbar) were present and enabled.  I had to disable them in about:addons;  I wanted to completely uninstall them but I couldn’t, the “uninstall” button just wasn’t present.  The Java Consoles and the media player were disabled because they were incompatible with 7.0.1, but I was also not able to  uninstall them.  This horrified me.  Is it a Windows-only behaviour?  Whatever the explanation, the default situation in a fresh install was that Firefox had several unnecessary, ugly additions, and it took some effort to remove them.  I’m really hoping that the add-on confirmation screen that has been added to Firefox 8 will help improve this situation, because this was the single worst part of the process.

I then tweaked the location of the home and reload buttons so they were in exactly the same position as in Chrome.  I probably didn’t need to do that, but  with those changes made Firefox’s UI looked very similar to Chrome’s, and I wanted things to be as comfortable for her as possible.

The best part of the process was when I installed AdBlock Plus.  With Karen watching, I visited in Chrome, and I had to skip past a video advertisement before even getting to the front page, and then the front page had heaps of ads.  Then I visited in Firefox — no video, no ads.  It was great!


A week or two later I sent Karen a follow-up email to check that everything was ok.  She said “All is well! The ad blocking is a great feature.”

So, Firefox has a new and happy user, but there were some obstacles along the way, and the outcome probably wouldn’t have been so good if Karen hadn’t had a Firefox expert to help her.


On vacation September 12 to October 17

I will be on vacation from September 12 to October 17.  During that time I will be online sporadically, if at all.

I won’t be attending the Mozilla all-hands meeting.  My vacation was scheduled before the all-hands dates were announced, unfortunately.

However, I will visit Mozilla HQ in Mountain View on September 27 and 28.  This means I’ll be present for the launch of Firefox 7 (the first release to benefit from MemShrink) and also for a MemShrink weekly meeting.

Firefox Personal

Mum ♥

My mother has been staying with me and my family this past week. I told her about the Firefox 4 release, and showed her

Firefox 4 download counter

She was mesmerized:  “There’s one in Japan… oh, one in Sydney… South America’s doing well… several in Russia…”  I’ve pulled my laptop out to show her it several times, and given her download counts multiple times a day: “7.1 million in the first day… 10 million… 22 million… 35.5 million…”  As a result, I’ve gone on to explain to her a lot more about Firefox, its history, competition between browsers, and how I contribute to Firefox.

I give a big thank you and congratulations to the metrics team for presenting some potentially dry statistics in a beautiful, compelling and easy-to-understand manner!


Zachary Blake Nethercote

Nick, Phoebe and Keira are pleased to announce the birth of Zachary Blake Nethercote, born on May 24, 2010, weighing 3.45kg (7lb 9oz).

Everyone is healthy and doing well!

Mac OS X Personal Valgrind

Me, Valgrind, and Mac OS X

Welcome to my blog, where I’ll be discussing some of the work I’m doing for Mozilla.

A little about me

I’m Australian. I live in Melbourne. I’ve also lived in Cambridge, England and Austin, Texas, and so I am fluent in at least three dialects of English. I like spending time with my wife Phoebe and baby daughter Keira, eating food, riding my bike, and following US presidential elections obsessively. Two weeks ago I left the academic/research world and started working for Mozilla.


My first big task for Mozilla is to improve support for Mac OS X in Valgrind. I’ve been involved with Valgrind since before the 1.0 release in 2002, and have done lots of work on it, including writing two tools that are in the Valgrind distribution: Cachegrind, a cache profiler, and Massif, a memory profiler. I even wrote a PhD dissertation about it.

And it seems that lots of Mozilla people find Valgrind useful, which is nice. However, it currently only runs on Linux. (Well, it also runs on AIX, but not many people care about that.)

Valgrind on Mac OS X

More than four years ago, on December 16, 2004, an Apple employee named Greg Parker wrote to the Valgrind developers mailing list to tell us that he was working on a port of Valgrind for Mac OS X.  He’s been working on it ever since then. (This must be why Mac OS 10.5 shipped late.)

After such a long time, I’m happy to report that there is now a branch holding Greg’s port in the Valgrind SVN repository.  If you want to check it out, do this:

  svn co svn:// <workspace-name>
  cd <workspace-name>

and then build it according to the instructions in the README file.  The branch is called DARWIN because Darwin is the name of the Mac OS “core”, which consists of a Mach-based microkernel and a few other bits and pieces.

However, please note that the port currently is, in Greg’s words: “UNSUPPORTED and INCOMPLETE and BUGGY… It may not find bugs in your program, or run your program correctly, or run your program at all.” What Greg has done is very impressive, and goes an awfully long way towards having a complete port of Valgrind on Mac OS X.  But it’s not the cleanest patch ever.  To give you an idea…

  • The patch I imported was 31,144 lines, just over 1MB of text.
  • The patch initially didn’t work on 32-bit Macs.
  • The patch broke Valgrind on Linux.  This took me a couple of days to fix, mostly involving the addition of appropriate #if statements.
  • The patch broke the regression test system;  they wouldn’t even build, let alone run. After fixing them to run again, more than half of the tests failed on Linux, and almost three-quarters failed on Mac.
  • There are lots of compiler warnings.  (The Valgrind trunk has none).
  • Much of the code in the patch has 4 space indenting;  the rest of Valgrind code has 3 space indenting.

So there’s plenty of work to be done to get the branch into a state where it will be suitable for merging with the trunk.  It’s hard to estimate how long this will take, it will just be a matter of fixing things one piece at a time.  My guess is that three months might suffice, but it’s really just a guess.  But here are some metrics I can use to judge progress, and their values just after I got the the system and regression tests building and running again on Mac and Linux:

  • The number of regression test failures on Linux: 477 tests, 220 stderr failures, 53 stdout failures, 25 post failures.  (“stderr” failures generally indicate that Valgrind’s output had a problem, “stdout” failures generally indicate that the test program’s output had a problem, and “post” failures indicate that the output of a Valgrind post-processing step had a problem.)  These numbers roughly indicate how much existing functionality has been broken on Linux by the Darwin changes, and should be fairly easy to get down.
  • The number of regression test failures on Mac:  419 tests, 293 stderr failures, 58 stdout failures, 29 post failures.  These numbers are the most important, as they roughly indicate how complete the Mac functionality is, and will be much more work to get down.
  • The number of compiler warnings: 186.  This number should be easy to reduce.   (Update, Jan 20: That’s on Linux. On Darwin it was 461.)
  • The size of the diff between the branch and the trunk: 55,852 lines, 1.9MB.  This is larger than the original patch because some files have been moved on the branch but not yet moved on the trunk, including some tests that are large and have large expected outputs.  This number will go down in fits and starts;  it will never get to zero, as the final merge will happen when there are many differences between the branch and trunk.

I’ll occasionally post updates to these numbers so people can track progress.

If Valgrind-on-Mac is of interest to you, please try out the new branch and let me know how it goes. Note that I’m working on an old MacBook Pro which is only 32-bit, so it’s possible that I’ve broken the 64-bit Mac support, but have no way to determine this.