moz-headless-screenshot

July 29th, 2010

On a personal project of mine, I have a need to generate thumbnails of web pages. Until recently my solution was to use Firefox running in Xvfb, a virtual X server. This is not an ideal solution, as it requires you to have lots of X client libraries installed on your server. Additionally, Firefox is not intended for this purpose, so there are lots of ways for things to go wrong.

Because of this, I’d been following Chris Lord‘s work on the offscreen branch of Mozilla for some time, but never tried it out until recently. The offscreen branch provides a widget backend for Mozilla that can render web content to an offscreen buffer. Chris wrote it in support of Clutter, which is a pretty neat use case. Conveniently, he also provided a sample embedding client application called moz-headless-screenshot. This is a simple command line tool that takes a URL, image size, and output filename and generates a PNG screenshot of the webpage. This being exactly what I wanted, and having my poorly-written Firefox+Xvfb solution fall apart due to a server migration, I decided to give his solution a shot.

I hit a few speed bumps on the way, since there wasn’t much documentation to be found on actually building and using moz-headless-screenshot. I’ve attempted to fix this my providing detailed steps and a Makefile in my own moz-headless-screenshot repository. I’ve also modified the code slightly such that it’s easier to run (at least in my use case). I have heard from others over the years that have this same need, so hopefully someone else finds it useful!

Build Config peers

July 29th, 2010

I have been remiss in blogging, but as of a few weeks ago, we have two new Build Config peers:

  • Mitchell Field <mitchell.field@live.com.au>
  • Kyle Huey <me@kylehuey.com>

Both are available for reviewing build system patches.

MozillaBuild 1.5

July 22nd, 2010

MozillaBuild 1.5 has been released:
http://ftp.mozilla.org/pub/mozilla.org/mozilla/libraries/win32/MozillaBuildSetup-1.5.exe

The major highlights include:

  • A newer Mercurial (1.5.4)
  • Support for Visual C++ 2010
  • A newer Python (2.6.5)

You can see the full list of dependent bugs,  as well as the full list of committed changes.

As usual, bugs can be filed at bugzilla.mozilla.org.

Isabella Mielczarek

April 12th, 2010

I’d like to announce the birth of our first child, Isabella Grace-Ann Mielczarek:

Isabella in her coming-home outfit

She was born at 1:12 PM on Saturday, April 10th, weighing in at 7 lbs 8 oz. Mom and baby are doing great, and we just brought her home this morning. There are more photos available on my website.

I’ll be taking paternity leave starting today and returning April 26th. In the mean time, if you need help with:

  • Build system/Breakpad symbols or processing issues—See Jim Blandy
  • Breakpad client issues—See Chris Jones
  • Unit testing issues—See Clint Talbert

Easy branch-landing of patches

December 2nd, 2009

I often find myself landing patches on our 1.9.2 and 1.9.1 branches, both of which are in Mercurial repositories. This generally involves getting a changeset from the mozilla-central repository into one of these repositories, and also amending the changeset message to include the name of the person who gave me approval to land. There was some discussion recently on how it’s kind of a pain to do that. I’ve cooked up an easy solution for my own needs, perhaps it will serve yours as well.

I use “hg transplant” to get changesets from one repository to another. This assumes that you have local clones of both the source repository (mozilla-central in this case) and the destination repository (mozilla-1.9.2 or 1.9.1, usually). Assuming you had both clones side-by-side in a directory, you could run the transplant command in the destination repository’s working directory like so (where xxx is the changeset identifier of the changeset you want transplanted, you can also specify more than one changeset):

hg transplant -s ../mozilla-central xxx

The transplant command conveniently includes a “–filter” option that will let you alter the commit message or patch while transplanting. This requires you to have some sort of script for transplant to run. Here’s what I’m using (on Linux):

#!/bin/sh

if test -n "$APPEND"; then
 echo " $APPEND" >> "$1";
else
 if test -n "$EDITOR"; then
 $EDITOR "$1";
 else
 editor "$1";
 fi
fi

Save this as “transplant.sh” somewhere (and ensure that it’s executable), then in your ~/.hgrc, add a section:

[transplant]
filter = /path/to/transplant.sh

Now, when you run “hg transplant”, by default it will open an editor to edit the commit message for each changeset, allowing you to add approval information. But, even better, transplant.sh will append the contents of the “APPEND” variable if set, so you can run transplant like so to quickly append approval information:

APPEND="a=someone" hg transplant -s ../mozilla-central xxx

I find that this saves me a bunch of time, so hopefully it’s useful to someone else!

Source Server, back on trunk

October 5th, 2009

Some time ago, Lukas Blakk implemented support for a source server on our Windows builds as a class project in Dave Humphrey‘s class at Seneca College. Of course, soon after that we switched our main VCS from CVS to Mercurial, which broke all of her hard work. Thankfully, we got another one of Dave’s students, Jesse Valianes, to fix things to make it work with Mercurial. We landed his patch, but as it turns out we never enabled a setting on our build machines to make it actually work. However, when we finally tried to do so, I found out that another patch we had landed in the interim had broken things. I finally landed a fix for that, and we flipped it back on, and so today’s trunk build is source-enabled again.

If you have no idea what any of this means, it means you can download a Windows nightly build, attach a debugger, have it download the debug symbols automatically from our symbol server, and the debugger will download the matching source for you automatically.

I hope to get this backported to our 1.9.2 and 1.9.1 branches ASAP, so that our 3.5.x and 3.6 release builds will be similarly debuggable.

Firefox Packaging

September 17th, 2009

I recently landed some changes (on trunk and 1.9.2) to the way Firefox packaging works. There are two immediate consequences of this you should be aware of:

  1. Mac builds now use a packaging manifest just like Windows and Linux. If you add a file that you intend to ship on Mac, it needs to wind up in a packaging manifest. (bug 463605)
  2. All the  packaging manifest files have been combined into one single file: browser/installer/package-manifest.in. This should save everyone some time and annoyance. (bug 511642)

These changes had no effect on applications other than Firefox.

MozillaBuild 1.4

July 24th, 2009

MozillaBuild 1.4 is now available for download. This release focused primarily on compatibility with systems running the x64 edition of Windows. It also contains a few other niceties such as a newer version of Mercurial and other tools, and support for all known Microsoft SDKs. You can view the full list of changes from version 1.3 here.

For a long time when our unit tests or Talos performance tests encountered a crash, the result was nothing but frustration. If you were lucky, you could tell that it crashed, but you had no idea where. Poor Blake spent weeks tracing down a crash from his speculative-parsing patch that only seemed to occur on Talos. Up until recently I figured the only way to make this happen was going to involve a fair amount of work that only I was going to be able to do. A few weeks ago it was determined that this was becoming a significant impact on development, as patches would get checked in, cause a crash and be backed out, leaving the developer with nothing to go on.

Benjamin Smedberg has been hard at work making it possible to get stacks in this situation, using the same Breakpad utilities we use on our Socorro server, but locally on the machine running the tests. Practically all of the pieces were in place this afternoon when #developers cornered Alice and closed the tree while she landed the final patch to make Talos produce stack traces. Boris then committed a test crash, and as a result we were able to see crash stacks in Mochitest (OS X, Linux) as well as Talos (OS X, Linux).

Thanks to Benjamin for doing most of the heavy lifting here, and for
Alice for taking the Talos part across the finish line. The Talos work
was mostly in bug 480577, and the unit test work was bug 481732. Note
that currently this only works in Mochitest (all 4 varieties), it will
work in Reftest/Crashtest after bug 479225 is fixed (which should be soon).

(Cross posted in dev.tree-management, but posting here for a wider audience.)

Getting There from Here

February 9th, 2009

The New Yorker has a great article about reforming the US health care system entitled “Getting There from Here.” It’s not terribly long and if you’re at all interested in the topic I’d recommend reading it. It discusses how other nations with universal health care arrived at their present systems, which is a topic that seems completely absent from debate on the subject in the USA. Anyway, I’m not really posting here to talk about health care, but one part of this article rang true with me in other ways:

There is no dry-docking health care for a few months, or even for an afternoon, while we rebuild it. Grand plans admit no possibility of mistakes or failures, or the chance to learn from them. If we get things wrong, people will die. This doesn’t mean that ambitious reform is beyond us. But we have to start with what we have.

As owner of the Mozilla build system, I hear a lot of complaints. This is understandable, our build system is showing its age, and we are certainly straining against the limits of Autoconf and GNU make on a regular basis. Along with complaints, I hear a lot of suggestions of the form “why don’t you just use X“, where X is any of a number of alternative tools such as CMake or SCons. The basic answer parallels the quote above. Because we don’t have time to stop and rewrite everything. Our build system contains tens of thousands of lines of makefiles, as well as a configure.in that’s over 8,000 lines. Converting this much build junk by hand is doomed to failure. Converting it through automated tools might be possible, but we would need a smart plan, and it would likely involve testing the tools in parallel with the existing build system for a while in order to make for a smooth transition. In any case, it’s clear that if we are to find a way forward, it will require building on what we have, and not burning it to the ground and starting over from scratch.

[via]