Axel Hecht Mozilla in Your Language

July 25, 2008

compare-locales 0.5 is out

Filed under: L10n,Mozilla — Tags: — Axel Hecht @ 7:35 am

I just uploaded compare-locales 0.5 on pypi. To update your current install, use

easy_install -U compare-locales

This is the new version that can handle mozilla-central. There are two main features that were required to do so:

  • Not get the directories to compare from client.mk.
  • Drop the default path for l10n directories.

The latter was already an optional parameter to compare-locales, and is now mandatory. This is mostly because with hg, you’ll more likely end up with several local repos representing branches or items of work. You just point which source to compare to which tree, and you’re done.

The first change is a bit more drastic. I created new files to hold the information about which directories actually contribute to the localization of an app, for Firefox on mozilla-central, cvs trunk, and the 1.8 branch, and for Thunderbird on cvs trunk and 1.8 branch. Now that cvs trunk is dead for shredder, we’ll need a new one on comm-central. I suggest to put those files into $(APP)/locales/l10n.ini, you can take a peek at the browser one on mxr. As you can see, it pulls the toolkit information in from a seperate ini file. If you’re creating an l10n.ini file for your app, feel free to CC me on the bug and request my review.

The main change for localizers is that you call it differently. It doesn’t really matter anymore from where you call it, just make sure that the first argument is the path to the l10n.ini, the second argument is the base dir for the localizations, and then list the locales that you want to compare. There is a remaining constraint in that the l10n base dir needs to have the localization in a subdir with the same name as the locale code.

compare-locales browser/locales/l10n.ini ../l10n de fr hi-IN

would compare German, French and Hindi Firefox localizations.

July 24, 2008

l10n buildbot update

Filed under: L10n,Mozilla — Tags: , — Axel Hecht @ 3:31 pm

I’ve updated the buildbot master and slaves running on the l10n server in the last few days.

There have been a few reasons to do so:

  • for Firefox 3.1 localization, I will need an updated compare-locales
  • … and support on the dashboard
  • there’s a new buildbot 0.7.8 around the corner
  • … which offers scheduler properties, which make l10n buildbots sooooo much nicer
  • and, hrm, I had bugs to fix in the history view

For a while now it was obvious that the code running the buildbot on l10n.mozilla.org would be much nicer and easier to grok, if it used the new features upcoming in buildbot, namely, scheduler properties. I’ve been doing some weird hacks to get around the lack of those on buildbot 0.7.7, and nobody liked those, including me.

Luckily, it wasn’t hard at all to drop all the weird code I used in favour of scheduler properties, for the most part, it was removing code. For the curious ones among you, have a look at the crucial diff.

The downside of that change is that I needed to update my custom code at a few places that used build properties in the 0.7.7 style. Once I learned the tricks, the patch was rather systematic, though, no real surprises. Again, there’s a patch in the original queue for curious folks.

I’ve followed the buildbot trunk darcs repository since the announcement of the imminent release last week on my local setup, and found a few issues. Which is cool, as those issues are fixed now, and I have a unpatched buildbot 0.7.8pre running on l10n.m.o. This is actually a real treat, as we (the release team as well as I) have been running slightly patched releases of buildbot to fix bustages in the past. And those bustage fixes tended to be slightly different upstream than in our repo, making upgrading to a new version of buildbot a pita.

On top of that work on the backbone, I’ve fixed a few timezone issues all around, which most apparently broke the statistics pages. They’re linked to from the dashboard as history (H). Those most apparently showed when trying to click on the bonsai links. There are blue vertical marker lines for l10n check-ins, and if you click on those, a lens opens with the commiter, the files and the check-in comment. The commiter email is actually a link to bonsai-l10n. UE guidance welcome.

While I was at it, I fixed both the start and end times of the timelines, as well as their display. Now they’re actually piecewise constant, as they should be. Ok, they’re not. They’re still P1, because timline likes that, but they look really close to P0.

Part of the preparations for fx3.1 was the renaming of “trunk” into “fx30x”. That should not only enable us to have fx3.1 on the dashboard, but other projects, too. Those required some changes to the existing data archive, in particular the database that I have locally. Those changes weren’t too bad, though.

Known regressions:

  • I check out from scratch now on each build. Should be less of an issue on hg. And CVS is stable, and I only check out the locale that I build. Thus resolved WHATEVER.
  • If anybody has links referring to “trunk”, those are broken now. “fx30x” is the new name for “trunk”.
  • If you find more, file a bug, comment, or drop me a mail.

July 22, 2008

white russian

Filed under: L10n,Mozilla — Tags: , , — Axel Hecht @ 3:38 pm

Where do we go, sings Marillion. In this context, rather, where do my thoughts go.

l10n builds it is again. I’m currently working on porting my buildbot work over to buildbot 0.7.8, and prep it for mozilla-central. There are a few interesting points:

  • 0.7.8 is not released yet. Finding bugs, fixing bugs, reporting back upstream.
  • 0.7.8 has scheduler properties. Drop all the fuzzyness about queues in schedulers etc. Just set up properties right away. Smoooooothness. Lost merging builds for now.
  • mozilla-central doesn’t have l10n configuration in client.mk. That’s bug 445217, landed on hg and various cvs branches. This change goes along with various changes to compare-locales, which I haven’t released yet.
  • As I don’t have to call into client.mk anymore and read the config data from plain files instead, I can do that asynchronously (more easily). My master just starts up now.

All in all, interactions inside the l10n build scheduling are much easier to grok thanks to scheduler properties.

To add some cream to the milk, here’s how I picture l10n builds to go one day:

  • pull central (and friends for Thunderbird, SeaMonkey)
  • pull and update to tip for locale to build
  • for the most performant platform:
    • update central and en-US to tip
    • compare-locales with rich reporting to dashboard and siblings
    • possibly other source verification and test code
  • update en-US to nightly changeset
  • compare-locales with merge, fail on error
  • repack, fail on error
  • upload binaries

The source analysis step is going to be the important one to be used for the dashboard, as that needs to have the most current information.

The actual builds on the other hand will be generated against the changesource for the last nightly, i.e., they will merge in en-US content based on the right data, and will only fail on rare occasions where the merge is harder than necessary, or when there are plain bustages in the localization.

In case you really want to know, the patches I’m currently working on are in a public hg queue repo.

July 3, 2008

l10n merge

Filed under: L10n,Mozilla — Tags: — Axel Hecht @ 10:10 am

I’ve just pushed an implementation of l10n-merge to my tooling repository. It’s now actually just an option to compare-locales, and will do the weakest heuristic for now.

Whenever compare-locales finds missing entries in an existing file, it will create a copy of that file in a staging directory for the merge, and append the missing entries. For the bulk of our files, that should work fine. Bookmarks.html is an exception, as are the netError files, I think order of entities and dtd inclusions matters there. In the end, you get a staging directory with files that got fixed up, a localization directory with both good files and files with missing entries, and the original en-US source. Making jar.mn actually pick localized files up from three different places in a particular order is in my build-patches repository.

Here’s why, in case you wonder: First and foremost, it leaves the original source alone. I like it like that. Secondly, it does as few file manipulations as possible in the best case, a complete localization doesn’t do a single copy or something. It’s not all that invasive into the build system as one might think, too. At least as soon as you want to look at the code to see where you’re looking for files, checking a bunch of source base dirs is rather trivial.

Powered by WordPress