Porcupine, meet Churchill

I’ve been talking with Seth today on how we can answer questions about the status of l10n. My grumpy argument was that I wouldn’t know how to make graphs over time actually show progress, instead of just “failure”. I had two naive graphs, one is showing all missing strings summed up over all locales. That graph would be dominated by the long tail of several dozen locales with a few hundred strings each, and you wouldn’t see a dozen fighting over a few strings each.

The other is what I nick-name “porcupine graph”, show how many locales have no missing strings, vs those that have some missing strings. This is what’s actually implemented on the l10n dashboard as tree progress graphs. But how ever small a string change would be, it goes to all red. And it doesn’t help that one can’t mix green and red color gradients, so the graph usually shows spikes of red and a little black.

porcupine

Who’d want that as their progress stats, huh?

Now, during the chat with Seth I came up with the idea to just give a little bit of leeway, and accept some missing strings to be OK, at least for some time. I filed bug 582280 on that, and made a rough initial implementation of it. Nothing fancy, just a constant ignored bound of missing strings. Let’s see how the past two weeks of Firefox 4 look now, with just a total of 5 missing strings being OK, ?bound=5:

two weeks good and bad

Now Churchill won over the porcupine, but it’s still pretty red. Which is OK, we haven’t even branched yet, right? So I went ahead and figured I’d add an option hideBad:

two weeks good

Wow, progress. This graph actually looks like our community rocks as much as it does. Gets me grumpy, because this was really just about half an hour of work, plus a few years of thinking.

Now, how do we look on the long run, say, well over half a year? Bumping the bound up to 15, we’re doing like

half year progress

Pretty good, heh? You can play with it on the dashboard, too. The overall take aways would be:

We have about 20 locales that really track trunk.

We didn’t have that many landings with a high amount of added strings.

I like both :-).

The Conversation {2 comments}

  1. Tony Mechelynck {Tuesday July 27, 2010 @ 8:06 pm}

    - What happened in the first half of the year, that we only recently came back to the number of “locales with 15 bad strings or fewer” that we had in January?

    - I wouldn’t call eight months “the long run”. Do we have statistics for ten years, or at least five?

  2. Axel Hecht {Wednesday July 28, 2010 @ 1:58 am}

    Regarding your first question, I don’t currently have a tool that analyses en-US landings. It “could” be written, but it’s not totally trivial to do so. The first speculation is, quite a few landings with significantly more than 15 new strings. Also, one sees a saturation of locales catching up after 2 weeks.

    Regarding your long-term stats, I’m ten years in Mozilla, and we’re doing l10n in version control for about 5, much less in hg. Your timeframe is a bit out of scope, IMHO ;-). That said, gathering this data for at least all-hg is quite a big computational task, which I didn’t go in to when I’ve set up the current iteration of the dashboard. Rerunning old data isn’t all trivial, due to some optimizations that sort on pk instead of dates, and other tidbits. If we ever get to the point that we have to really regenerate lots of data inside the dashboard, I might, with hardware and time, start the data computation when we started with hg. I will most likely not go and create a system that can deal with CVS to generate the same data.

Sorry, comments for this entry are closed at this time.