A lot of improvements to Mozilla l10n tools and process have come throughout 2016. We’ve deeply enjoyed connecting with members of the community to learn what other improvements you would like to see in l10n. For the rest of the year, the l10n team has a number of goals we’re working toward accomplishing by 1 January 2017:
Land L20n in Firefox desktop,
Implement cross-channel localization for Firefox and Firefox for Android,
Track all project statuses in GitHub Projects and report frequently,
The last weekend of August saw the largest localization hackathon event the l10n-drivers ever organized. Thirty-four community contributors representing 12 languages from 13 East and Southeast Asian countries journeyed to Kuala Lumpur, Malaysia on Friday, August 26. Jeff, Flod, Gary Kwong and I arrived in time for the welcome dinner with most of the community members. The restaurant, LOKL Coffee, was ready for a menu makeover and took the opportunity to use this Mozilla event to do just that. A professional photographer spent much of the evening with us snapping photos.
We started off Saturday morning with Spectrogram, where l10n contributors moved from one side of the room to another to illustrate whether they agreed or disagreed with a statement. Statements help us understand each community’s preferences to address localization requests. An example: There are too many translation/localization tasks for me to keep up; I want to work on 2000 strings sliced up in 1 year, twice, 6 weeks, 4 weeks, weekly, every other day, daily.
Jeff, the newly appointed localization manager, updated everyone on l10n organization change; the coming attraction of the l20n development; Pontoon as one of the centralized l10n tools; and the ultimate goal of having a single source of l10n dashboard for the communities and l10n project managers.
Flod briefed on the end of Firefox OS and the new initiatives with Connected Device. He focused on Firefox primarily. He discussed the 6-week rapid release cycles or cadence. He also covered the five versions of Firefox: Aurora, nightly, beta, release, and ERS. He described the change to a single source of repository, allowing strings move to production sooner. Firefox for iOS and Android were also presented. It was welcome news that the localized product can be shipped through automatic signoff, without community’s involvement.
I talked about the importance of developing a style guide for each of the languages represented. This helps with onboarding new comers, consistency among all contributors and sets the style and tone for each of the Mozilla products. I also briefly touched upon the difference between brand names and product names. I suggested to take this gathering as an opportunity to work on these.
For the rest of the weekend, our communities worked through the goals they set for ourselves. Many requested to move their locales to Pontoon, causing a temporarily stall in sync. Others completed quite a few projects, making significant advances on the dashboard charts. Even more decided to tackle the style guides, referencing the template and leveraging information from established outlets. When the weekend was over, nine communities reported to have some kind of draft versions, or modified and updated an existing one. Other accomplishments included identifying roles and responsibilities; making plans for meetup for the rest of the year; tool training; improving translation quality by finding critical errors; updating glossaries; completing some high priority projects.
The weekend was not just all work, but filled with cultural activities. Our Saturday dinner at Songket Restaurant was followed by almost an hour of Malaysian cultural dances from across the country, showcasing the diverse cultures that made up Malaysia. Many community members were invited to the stage to participate. It was a fun evening filled with laughter. Our Sunday dinner was arranged inside Pasar Seni, or the Central Market, a market dating back to 1888. It is now filled with shops and restaurants, giving all visitors a chance to take home some souvenirs and fond memories. Many of us visited the near by Pedaling Street, sampling tropical fruits, including Durian, made in all shapes and forms.
Putting together the largest l10n hackathon ever is a big achievement and lots of credit goes to our local support. A big thanks to our Malaysian community, led by Syafiq, who was our eyes and ears on the ground from day one, planning, selecting the venue location, advising us on restaurants, lodging, transportation and cultural events. Not only we accomplished what we set out to do, we did it safely, we all had fun and we made more friends. Also a shout-out to Nasrun, our residence photographer for documenting the weekend through his lens. And a thank you to everyone for sharing a very special and productive weekend with fellow Mozillians! See you next time at another hackathon!
One of the main goals of Pontoon is lowering barriers to entry. Especially for end users (mainly localizers), but also for contributors to the codebase, since many of our localizers have a developer background.
I’m happy to acknowledge that in the last 30 days there has been more activity from volunteer contributors in Pontoon development than ever before! Let’s have a closer look at what have they been working on:
Last month’s Pontoon contributors
Michal came up with the idea to highlight matches in original and translated strings when searching in the sidebar. He created a patch, but couldn’t finish it due to his school duties. It was taken over by Jarek, who earlier played a great role in reviewing the original patch by Michal.
Being only 14 years old, Michal is the youngest Pontoon contributor!
Jarek Śmiejczak (jotes)
Since he became an active Pontoon contributor over a year ago, Jarek has evolved from being not just a great developer but also a fantastic mentor; helping onboard new contributors and review their work. One way or another, he’s been involved with all bugs and features listed in this blog post.
Of course that doesn’t mean he stopped contributing code. On the contrary, he just completed a Firefox Accounts based authentication support which will soon replace Persona. And, he’s already busy working on bringing terminology support to Pontoon too.
As the new leader of the Bulgarian localization team, Stoyan takes his duties very professionally. He started by creating a vector version of Pontoon logo and changing the copy.
Later on he created a Firefox Add-On called Pontoon Enhanced, which can add new features to Pontoon before they are deployed or even implemented in the application. It’s basically a Test Pilot for Pontoon.
As an agile bug reporter, Michal has been one of the most valuable early adopters of Pontoon. Now he has decided to take a step further.
He set up his local Pontoon instance, fixed a few developer documentation bugs along the way and provided a patch that fixes one of the bugs he reported. It allows us to properly detect placeables of form %(thisIsVariable)s.
I consider myself lucky to be working with this great team. It is particularly valuable to see contributions coming from people who actually use the product. This is what the power of the open Web looks like!
Here’s an outline of what is currently in Aurora this cycle for Firefox 50 and some information on the accomplishments of the l10n Community during the previous cycle.
Current Aurora Cycle – Firefox 50
Key dates for this cycle:
Beta (49): localization updates for already shipping locales must be completed before 31 Aug. For reference, the date is roughly 2 weeks before the next release date, and it’s the last good day to include your updates into a Beta build.
Aurora (50): localization updates must be completed before 12 September. That’s the Monday, also known as merge day, before the next release of Firefox.
Firefox Aurora desktop has 155 added strings (125 obsolete). About 43% of the new strings are for Developer Tools.
Fennec Aurora has 88 new strings (61 obsolete). 11 new strings are Fennec-only (in /mobile).
There are currently no pending requests to uplift patches with strings to Aurora.
For further details on the new features you can check the release notes (they’re usually published a few days after release):
Noteworthy events for Firefox 48 (release date 3 Aug):
52 locales, corresponding to 63% of our shipping locales, signed off updates for Firefox 48 on desktop.
48 locales, corresponding to 72% of our shipping locales, signed off updates for Fennec 48 on Android.
As recently announced, localizers are not going to request sign-offs anymore for Firefox and Fennec, l10n-drivers will sign-off and review any update landing in the repository, for both Aurora and Beta.
For this reason we’re probably not going to include this section in the next cycle reports, while we determine more meaningful metrics to track localization activity.
Noteworthy Changes Available in Aurora
These are some of the interesting changes introduced in the last cycle.
The new Containers feature is not enabled outside of Nightly (it might be included in a future Test Pilot experiment). If you want to test it, you need to manually switch the preference privacy.userContext.enabled to True.
As explained in the previous report, team is starting to use a new way to define keyboard shortcuts (not accesskeys), adopting a syntax similar to Electron.
You should not translate fragments like “CmdOrCtrl”, “CmdOrCtrl+Plus” (Plus indicates the ‘+’ key), “CmdOrCtrl+Shift+D”. Also a reminder that you should not be changing shortcuts in general, unlike accesskeys, unless the default keyboard layout for your locale doesn’t include that specific key, or combination of keys. Translating these keys will result in the tools being broken.
Several developer tools are also moving strings from .DTD to .properties, it should be expected to have a perfect match in TM tools like Pontoon & Pootle between old and new strings. For example:
Translated reference entities and untranslated labels
In .DTD files, a string can contain a reference to another entity in the form of &another_entity_name; (note the ampersand at the beginning and the semicolon at the end). These are references to other string IDs and should not be translated.
Example from Fennec:
You can always turn this off in &settings; under &pref_category_general;.
A few locales translated the first &settings;, generating an error. It’s also good to remember that these errors break the multi-locale Android build for all locales, en-US included, and that’s why you will see someone from l10n-drivers committing a fix directly in tools (Pontoon, Pootle) or Mercurial. Please remember to keep an eye on the dashboard page for your locale and verify errors and warnings.
On the other hand, Pootle displays a string with an accesskey as &Show: the label is “Show”, the accesskey is “S” (the character after the ampersand). Note that there’s no semicolon at the end. This has to be translated, and dropping the ampersand will make the accesskey fallback to English. Never add the accesskey to your label, e.g. “Show (S)”. For further details about accesskeys, see this discussion on dev-l10n.
Thanks to everyone for your dedication and hard work this last cycle. If you note anything missing in these reports, or would like to see other information included, please let me know.
L20n is a new localization framework for Firefox and Gecko. Here’s what you need to know if you’re a Firefox front-end developer.
Gecko’s current localization framework hasn’t changed in the last two decades. It is based on file formats which weren’t designed for localization. It offers crude APIs. It tasks developers with things they shouldn’t have to do. It doesn’t allow localizers to use the full expressive power of their languages.
L20n is a modern localization and internationalization infrastructure created by the Localization Engineering team in order to overcome these limitations. It was successfully used in Firefox OS. We’ve put parts of it on the ECMA standardization path. Now we intend to integrate it into Gecko and migrate Firefox to it.
Overview of How L20n Works
What problems L20n solves?
The current localization infrastructure is tightly-coupled: it touches many different areas of the codebase. It also requires many decisions from the developer. Every time someone wants to add a new string they need to go through the following mental checklist:
Is the translation embedded in HTML or XUL? If so, use the DTD format. Be careful to only use valid entity references or you’ll end up with a Yellow Screen of Death. Sure enough, the list of valid entities is different for HTML and for XUL. (For instance …
is valid in HTML but not in XUL.)
Does the translation depend on a number in any of the supported languages? If so, use the PluralForm.jsm module to choose the correct variant of the translation. Specify all variants on a single line of the .properties file, separated by semicolons.
Does the translation comprise HTML elements? If so, split the copy into smaller parts surrounding the HTML elements and put each part in its own translation. Remember to keep them in sync in case of changes to the copy. Alternatively write your own solution for replacing interpolation specifiers with HTML markup.
What a ride! All of this just to add a simple You have no new notifications message to the UI. How do we fix this tight-coupled-ness?
L20n is designed around the principle of separation of concerns. It introduces a single syntax for all use-cases and offers a robust fallback mechanism in case of missing or broken translations.
Let’s take a closer look at some of the features of L20n which mitigate the headaches outlined above.
In addition to DTD and .properties files Gecko currently also uses .ini and .inc files for a total of four different localization formats.
L20n introduces a single file format based on ICU’s MessageFormat. It’s designed to look familiar to people who have previous experience with .properties and .ini. If you’ve worked with .properties or .ini before you already know how to create simple L20n translations.
Fig. 1. A primer on the FTL syntax
A single localization format greatly reduces the complexity of the ecosystem. It’s designed to keep simple translations simple and readable. At the same time it allows for more control from localizers when it comes to defining and selecting variants of translations for different plural categories, genders, grammatical cases etc. These features can be introduced only in translations which need them and never leak into other languages. You can learn more about L20n’s syntax in my previous blog post and at http://l20n.org/learn. An interactive editor is also available at https://l20n.github.io/tinker.
Separation of Concerns: Plurals and Interpolation
In L20n all the logic related to selecting the right variant of the translation happens inside of the localization framework. Similarly L20n takes care of the interpolation of external variables into the translations. As a developer, all you need to do is declare which translation identifier you are interested in and pass the raw data that is relevant.
Fig. 2. Plurals and interpolation in L20n
In the example above you’ll note that in the BEFORE version the developer had to manually call the PluralForm API. Furthermore the calling code is also responsible for replacing #1 with the relevant datum. There’s is no error checking: if the translation contains an error (perhaps a typo in #1) the replace() will silently fail and the final message displayed to the user will be broken.
Separation of Concerns: Intl Formatters
L20n builds on top of the existing standards like ECMA 402’s Intl API (itself based in large part on Unicode’s ICU). The Localization team has also been active in advancing proposals and specification for new formatters.
L20n provides an easy way to use Intl formatters from within translations. Often times the Intl API completely removes the need of going through the localization layer. In the example below the logic for displaying relative time (“2 days ago”) has been replaced by a single call to a new Intl formatter, Intl.RelativeTimeFormat.
Fig. 3. Intl API in use
Separation of Concerns: HTML in Translations
L20n allows for some semantic markup in translations. Localizers can use safe text-level HTML elements to create translations which obey the rules of typography and punctuation. Developers can also embed interactive elements inside of translations and attach event handlers to them in HTML or XUL. L20n will overlay translations on top of the source DOM tree preserving the identity of elements and the event listeners.
Fig. 4. Semantic markup in L20n
In the example above the BEFORE version must resort to splitting the translation into multiple parts, each for a possible piece of translation surrounding the two <label> elements. The L20n version only defines a single translation unit and the localizer is free to position the text around the <label> elements as they see fit. In the future it will be possible to reorder the <label> elements themselves.
Resilient to Errors
L20n provides a graceful and robust fallback mechanism in case of missing or broken translations. If you’re a Firefox front-end developer you might be familiar with this image:
Fig. 5. Yellow Screen of Death
This errors happens whenever a DTD file is broken. The way a DTD file can be broken might be as subtle as a translation using the … entity which is valid in HTML but not in XUL.
In L20n, broken translations never break the UI. L20n tries its best to display a meaningful message to the user in case of errors. It may try to fall back to the next language preferred by the user if it’s available. As the last resort L20n will show the identifier of the message.
L20n allows us to re-think major design decisions related to localization in Firefox. The first area of innovation that we’re currently exploring is the experience of changing the browser’s UI language. A runtime localization framework allows the change to happen seamlessly on the fly without restarts. It will also become possible to go back and forth between languages for just a part of the UI, a feature often requested by non-English users of Developer Tools.
Another innovation that we’re excited about is the ability to push updates to the existing translations independent of the software updates which currently happen approximately every 6 weeks. We call this feature Live Updates to Localizations.
We want to decouple the release schedule of Firefox from the release schedule of localizations. The whole release process can then become more flexible and new translations can be delivered to users outside of regular software updates.
L20n’s goal is to improve Mozilla’s ability to create quality multilingual user interfaces, simplify the localization process for developers, improve error recovery and allow us to innovate.
The migration will result in cleaner and easier to maintain code base. It will improve the quality and the security of Firefox. It will provide a resilient runtime fallback, loosening the ties between code and localizations. And it will open up many new opportunities to innovate.
After much delays, collectively we picked a balmy first weekend of June and Berlin as our host city for a localization hackathon. We had four representing each of Dutch/Frisian and Ukrainian communities, three of German, one of South African English. Most of them had not been to an l10n hackathon, many have never not met in person within the community even though they had been collaborating for years.
As with the other hackathons this year we allowed each team to plan how they spent their time together, and set team goals on what they wanted to accomplish over the weekend. The localization drivers would lead some group discussions. As a group, we split the weekend covering the following topics:
A series of spectrograms where attendees answer yes/no, agree/disagree questions by physically standing on a straight line from one side of the room to the other. We learned a lot about our group on recognition, about the web in their language, and about participation patterns. As we’re thinking about how to improve localization of Firefox, gaining insights into localizers hearts and life is always helpful.
Axel shared some organizational updates from the Orlando All-Hands: we recaped the status of Firefox OS and the new focus on Connected Devices. We also covered the release schedule of Firefox for iOS and Android.
We spent a bit more time talking about the upcoming changes to localization of Firefox, with L20n and repository changes coming up. In the meantime, we have a dedicated blog post on l20n for localizers, so read up on l20n there. Alongside, we’ll stop using individual repositories and workflows for localizing Firefox Nightly, Developer Edition, Beta, and release. Instead the strings needed for all of them will be in a single place. That’s obviously quite a few changes coming up, and we got quite a few questions in the conversations. At least Axel enjoys answering them.
Our renewed focus on translation quality that resulted in development of the style guide template as a guideline for localization communities to emulate. We went through all the categories and sub-categories and explained what was expected of them to elaborate and provide locale specific examples. We stressed the importance of having one as it would help with consistency between multiple contributors to a single product or all products and projects across the board. This exercise encouraged some of the communities who thought they had a guide to review and update, and those who didn’t have one to create one. The Ukrainian community created a draft version soon after they returned home. Having an established style guide would help with training and on boarding new contributors.
We also went over the categories and definitions specified in MQM. We immediately used that knowledge to review through live demo in Pontoon-like tool some inconsistencies in the strings extracted from projects in Ukrainian. To me, that was one of the highlights of the weekend: 1) how to give constructive feedback using one of the defined categories; 2) Reoccurring type of mistakes either by a particular contributor or locale; 3). Terminology consistency within a project, product or a group of products, especially with multiple contributors; 4) Importance of peer review
For the rest of the weekend, each of the community had their own breakout sessions, reviewed their own to-do list, fixed bugs, completed some projects, and spent one on one time with the l10n drivers.
We were incredibly blessed with great weather. The unusually heavy rain that flooded many parts of Germany stopped during our visit. A meetup like this would not be complete without experiencing some local cultures. Axel, a Berlin native was tasked to show us around. We walked, walked and walked and with occasionally public transportation in between. We covered several landmarks such as the Berlin Wall, the Brandenburg Gate, several memorials, the landmark Gedächtniskirche as well as parks and streets crowded with the locals. Of course we sampled cuisines that reflected the diverse culture that Berlin had been: we had great kebabs and the best kebabs, Chinese fusion, the seasonal asparagus and of course the German beer. For some of us, this was not the first Berlin visit. But a group activity together, with Axel as our guide, the visit was so much memorable. Before we said goodbye, the thought of next year’s hackathon came to mind. Our Ukraine community had volunteered to host it in Lviv, a beautiful city in the western part of the country. We shall see.
Earlier this week I came back from the Ljubljana Localization Hackathon which took place over the weekend. It was an inspiring meetup focused on translating Mozilla projects. I left full of energy and ideas and happy to have met many amazing people contributing to Mozilla.
Stas and I met in Riga with the Baltic and Polish l10n communities to do a hackathon. We gathered Latvian, Lithuanian, and Polish. Sadly, nobody from the Estonian community could attend due to conflicting schedules, but Merike Sell joined us on Saturday morning via Skype. We also had Dainis Šantars and Rūdolfs Mazurs join for a few hours each to work on Latgalian.
On Friday evening, we kicked things off warm and dry. That’s worth noting, as there wasn’t any weather like that for the rest of the weekend. Anyway, we grabbed some dinner, and ventured a few places to get a first feeling for Riga.
Saturday morning we met at the University of Latvia, and started the actual hackathon with the obligational introductions, and some “spectrograms”. We got some insights on testing, and which versions of Firefox our community uses. Reminder, Developer Edition or Nightly are the right versions. Use the version you work on. We also talked about the web and software in local languages. Seems that using localized versions becomes normal. We also got more insights into contribution patters that people like.
I really like the conversations about contribution patterns. We start the discussion with the question on how often people would like to localize. The answers quickly lead into discussions about localization quality, and which context localizers prefer to get that. But also to the role that Mozilla plays in people’s lives. Getting a better understanding of both are critical as we’re trying to make localization at Mozilla better.
Afterwards, stas and I gave some general project updates. The Mozilla firehose is overwhelming, so it’s good to give an update on the things that matter to the people in the room. In particular, stas covered L20n, which was well received. I talked about how we want to do Firefox l10n less bound to aurora and beta release channels, which people also liked.
We also talked about MQM, a standard of classifying issues found around localization. Once again we learned that it’s hard to explain them.
As usual, we let people work within their teams for half the time. We did help out with accounts, and persuaded folks to do reviews of suggestions. We heard they also used the time to strategically set up their team, and to learn from each other.
Last but not least, huge thanks to Raivis Dejus for organizing this event. He made us all feel welcome, and as warm as it gets, and always had an anecdote about Riga to share. He turned out to be my inspiration for the Berlin hackathon.
What is happening?
We are changing the way we are doing sign-offs. I REPEAT, we are changing the way we are doing SIGN-OFFS! 😉
What does it mean?
Basically, we are simplifying your life. In fact, you will not need to request sign-offs anymore.
We have realized that with time, this has become an unnecessary task that has lost its initial meaning. Sign offs, in the sense that we need to tell our system what translations go into a build, are still a technical necessity for the near future. Some of us l10n-drivers (Jeff, flod and I) will start managing the whole sign-off process allowing you to focus on your localizations.
How will it work?
The only thing that changes in your usual workflow is that you no longer need to request sign-offs. Essentially, as long as there’s a good, clean changeset, we will sign-off on it. From now on (we’ve already started), the designated l10n-drivers will perform sign-off reviews on all new changesets a few times a week. This will be the case on mozilla-aurora and mozilla-beta channels (Firefox Desktop and Firefox for Android). We will determine if the changeset is technically correct and will not break anything. You might also receive emails from us with notes about the string changes in case we find any issues.
We hope that this will result in shipping more good l10n updates to users. We consider that getting a localization update to a product – even if not complete – is better than no updates at all. True to our new motto “Simplicity and Opportunity”, we also believe this will simplify things on your side, and is a total win-win situation.
As usual, please feel free to reach out to us with any questions about this. Feedback as you know is welcome as well.