Translation quality at Mozilla

Jeff Beatty

2

Measuring translation quality is a shared priority

Part of what makes Mozilla projects so unique is the involvement of the community. Our community prides themselves on being experts in the Web, Mozilla, and all of Mozilla’s products. Thus, delivering high quality localizations of Mozilla products to users is not only a priority for the l10n-drivers, but one that is close to the community’s heart.

Currently, Mozilla has no criteria-based framework for evaluating a localization/translation’s accuracy or quality. Determining how to evaluate translation quality is a very difficult task because language itself is very flexible and subjective. Critical to creating a successful framework for evaluating translation quality is including elements of a project’s scope as well as the most objective pieces of language, such as orthography, grammar, and corporate style guide. A translation quality assessment framework would need to be flexible, robust, interoperable, and easy for graders to use. Developing such a framework is difficult, but I believe we’ve experienced a breakthrough!

Background of the MQM standard: Mozilla pilot projects

Translation Studies researchers from Brigham Young University (along with other North American and European organizations) set out to create a translation quality standard that would be flexible enough to accomodate project specifications and the needs of individual langauges. The Multidimensional Quality Metric (MQM) framework was born!

The MQM framework allows an organization to identify their highest priorities in assessing the translation quality of a specific project, or series of projects. The framework’s issue types are determined by the project’s organization. Graders for the project’s translations are recruited and trained on the meaning of these issue types and the process of grading a translation according to the selected issue type framework. Graders spend between 30-60 minutes per day marking the translation errors they find and categorizing them by issue type. The translation is then assigned a “score” according to the combined grades issued by the graders.

Delphine and I collaborated with these researchers to create MQM evaluation sprints for Firefox and Firefox OS localizations. Together, we found language and subject matter experts from within the university and the community to participate in these sprints. There were a total of three sprints organized over the last six months, each evaluating different languages and utilizing a variety of issue types. Participants were involved both virtually and in a physical space at Brigham Young University and were able to collaborate closely using IRC. Each individual grader evaluated between 400 – 4,000 strings in a single week!

Results of those projects

The results from these sprints were very positive. Localization teams were able to receive specific, actionable feedback on where their translations could be improved. This feedback was organized by issue type and called out specific strings as needed correction in one or more of the issue type categories. Some teams, like the French l10n team, were able to act immediately on that feedback and incorporated into their Firefox OS localization.

Community perception of MQM

Surveying the members of the community concerning their experience with the MQM framework returned very positive results. Generally speaking, localizers enjoyed how thorough the framwork’s definition was for the sprints and felt that the process was easy to understand. Many felt that the experience was valuable to their l10n work and were in favor of the l10n-drivers implementing the standard across the Mozilla l10n program.

Plans to implement MQM framework

There are preliminary plans to develop a Mozilla l10n QA tool based on the MQM framework. Ideas have included a gamified system, allowing users to choose to grade by project or by isolated issue type, and a sleak, web-based GUI complete with product screenshots for each string translation being evaluated. Some of the community has expressed interested in being involved in creating such a tool. We’ll be incorporating their feedback and getting their help as we move forward designing this MQM-based tool.

The MQM standard is governed by ASTM International. Mozilla has been invited to participate in the governing technical committee for the MQM standard, and we’re currently evaluating what our involvement in that committee would look like.

If you have any questions concerning the MQM framework at Mozilla, please feel free to email the Mozilla L10n mailing list.

Firefox L10n Report (cycles 31 & 32)

Jeff Beatty

0

Hello localizers!

Thank you all for your great work with Firefox 30 and 31. Here’s an outline of what is currently in Aurora this cycle (32) and what we accomplished together last cycle:

This cycle (Fx32) — 10 June – 21 July

Key dates:
- Beta sign offs must be completed before 14 July.
- Aurora sign offs must be completed before 21 July.
- Firefox 30 releases 22 July.

Features:
- Approximately 125 new string changes were made to Aurora desktop, 51 for Aurora mobile exclusively (unshared).
- 49% of the desktop strings changes are strings or files that need to be removed from your repos, including all of the metro files. 42% of all string changes are in devtools. The rest cover changes in form validation messages, the automatic translation feature, and others. (see https://wiki.mozilla.org/Features/Release_Tracking#Likely_in_Firefox_32 ).
- 24% of the mobile string changes are for getusermedia. The rest include locale switching, home page preferences, and message retrieval (see https://wiki.mozilla.org/Mobile/Roadmap#Firefox_32_.28Aurora.29 ).

Notes:
Please remember that sign offs are a critical piece to the cycle and mean that you approve and can vouch for the work you’re submitting for shipment.

Because nearly 50% of all 89 locales have not regularly signed off, and thus not shipping up-to-date localizations, I will begin contacting those teams this cycle to evaluate whether they can come up-to-date in a short amount of time, or if they need to be removed from the public Firefox release page ( https://www.mozilla.org/en-US/firefox/all/ ).

Last cycle — 28 April – 9 June

Noteworthy accomplishments:
- 69% of all locales shipped Firefox 30 on desktop updates on time. Congratulations to everyone who signed off and shipped this last cycle! This is an 12% increase in locale coverage between Firefox 29 and Firefox 30!
- 88% of all locales shipped Fennec 30 on time. Congratulations to everyone who signed off and shipped this last cycle! This is an 8% increase in locale coverage between Firefox 29 and Firefox 30!
- The Spanish (Mexico), Spanish (Argentina), Belarusian, Indonesian, Malay, and Latvian teams launched their first localizations of Fennec! Please contact the team to congratulate them on this massive accomplishment, and feel free to tweet all about!
- The new language switching feature in Fennec allows us to ship more locales on Fennec than ever before. If your locale wants to ship a Fennec localization, please contact us.

Thank you to everyone for all of your dedication and hard work this last sprint. As always, if you note anything missing in these reports, please let us know.

Language Switching in Fennec

Jeff Beatty

0

Allowing a user to select their language preferences for Android apps is typically tied to the operating system. Whatever languages your device lists in its OS language settings will determine the languages in which localization your apps, like Firefox for Android (also know as Fennec), will use. Essentially, if your language is not supported by Android, you’re unable to access your apps in your language either.

Additionally, the linguistic experience can be very limiting for the global Android end-user. Frequently device carriers and the manufacturers will filter which languages are available to the user to switch between. An Android device may be marketed as supporting 85 languages, but a user’s carrier in a particular region may only expose five of those 85 languages to the end-user.

These language support and filtration issues (among others) have made unlimited shipping of Fennec localizations an impossibility. We’ve had to carefully monitor the market to identify which languages are Android supported by device and region of the world before we could work with the regional localization team to ship their Fennec localizations. Many Mozilla l10n teams have kept Fennec localizations up-to-date since Fennec’s launch, waiting for a time in which their language will be shipped with Fennec. Thanks to the efforts of Richard Newman and the Fennec engineering team, this will no longer be the status quo for Fennec.

In Fennec 32, users will be able to select from all officially shipped localizations of Fennec within the browser itself. Languages no longer have to be among the list of Android supported languages to become official localizations of the browser. This moves us a huge step forward in being able to deliver Fennec in all of the languages of the world.

We’re still exploring how this will change any part of the onboarding new locales process for Fennec. When working on shipping a new locale, we’ll need the l10n teams’ help to test early and often for character rendering issues, input issues, UI restrictions, and other common internationalization and localization issues. However, if you have consistently maintained your Fennec localization and would like your work to be added to the official release, please send me an email and I will add you to the roadmap.

If you’re anxious to see this in action, this new feature can already be tested in Fennec 32 on the Nightly channel within the browser settings.

Screenshot_2014-05-20-09-55-38

 

Firefox Aurora L10n Report (cycles 29 & 30)

Jeff Beatty

0

Hello localizers!

Thank you all for your great work with Firefox 28 and 29. Here’s an outline of what is currently in Aurora this cycle (30) and what we accomplished together last cycle:

This cycle (17 March – 28 April)

Key dates include:
- Beta sign offs must be completed before 21 April.
- Aurora sign offs must be completed before 28 April.
- Firefox 29 releases 29 April.

Features:
- Approximately 94 new string changes were added to Aurora desktop, 57 for Aurora mobile.
- 34% of the desktop strings changes are in devtools, 6% are in metro, and about 14% are in DOM. The rest cover changes in preference UI for proxy autologin, GetUserMedia, Accounts (formerly Sync), and toolkit/mozapp/webapprt. (see https://wiki.mozilla.org/Features/Release_Tracking#Likely_in_Firefox_30 ).
- 24% of the mobile string changes are in DOM. Many of the mobile strings include context menu, home page preferences, and Firefox Accounts (see https://wiki.mozilla.org/Mobile/Roadmap#Firefox_30:_.28Beta.29 ).

Notes:
Please remember that sign offs are a critical piece to the cycle and mean that you approve and can vouch for the work you’re submitting for shipment.

This is a reasonably small cycle. If you have fallen behind (apprx 41% have), this would be a good cycle to catch up.

Last cycle (3 February – 17 March)

Noteworthy accomplishments:
- 59% of all locales shipped Firefox 26 on desktop updates on time. Congratulations to everyone who signed off and shipped this last cycle! This is an 11% decrease in locale coverage between Firefox 27 and Firefox 28!
- Estonian launched in Firefox for Android with Firefox 28. If your locale is ready for shipping and part of Android’s list of supported locales, please reach out to me to add you to the roadmap. Please join me in congratulating the team and sharing in your networks!

Thank you to everyone for all of your dedication and hard work this last sprint. As always, if you note anything missing in these reports, please let us know.

Firefox Aurora L10n Report — Aurora 28

Jeff Beatty

0

Hello localizers!

Thank you all for your great work with Firefox 27 and 28. Here’s an outline of what is currently in Aurora this cycle (29) and what we accomplished together last cycle:

This cycle (3 February – 17 March)

Key dates include:
- Beta sign offs must be completed before 10 March.
- Aurora sign offs must be completed before 17 March.
- Firefox 28 releases 18 March.

Features:
- Approximately 389 new string changes were added to Aurora desktop, 200 for Aurora mobile.
- Many of the new desktop strings are in devtools and metro. They cover the implementation of Australis, devtools, and first run in Metro (see https://wiki.mozilla.org/Features/Release_Tracking#Desktop_.28Win.2C_Mac.2C_Linux.2C_Metro.29_2).
- Many of the mobile strings include preferences and renaming of all Sync strings for Firefox Accounts (see https://wiki.mozilla.org/Mobile/Roadmap#Firefox_29:_.28Aurora.29 ).

Notes:
Please remember that sign offs are a critical piece to the cycle and mean that you approve and can vouch for the work you’re submitting for shipment.

Last cycle (9 December – 3 February)

Noteworthy accomplishments:
- 70% of all locales shipped Firefox 26 on desktop updates on time. Congratulations to everyone who signed off and shipped this last cycle! This is an 8% increase in locale coverage between Firefox 26 and Firefox 27!
- Aragonese and Xhosa had their first official release on Firefox desktop today with Firefox 27! Please join me in congratulating the teams!
- South African English, Thai, Lithuanian, and Slovenian were launched in Firefox for Android with Firefox 27. If your locale is ready for shipping and part of Android’s list of supported locales, please reach out to me to add you to the roadmap. Please join me in congratulating the teams!

Thank you to everyone for all of your dedication and hard work this last cycle.

Coining new terms for Firefox localization

Jeff Beatty

0

Every translator faces an important dilemna when translating text, whether it be a document, multimedia project, or strings in a piece of software: if a term in the source does not exist in my language, do I foreignize my translation (leave the term in the source language) or do I domesticate my translation (coin a new term or provide additional info in the target language). For some cultures, using source terms in the target language is common. In Mexico, for instance, the term computadora is a clear adaptation of the English source word computer. For other cultures, there is a high priority on coining new terms in the target language. In Iceland, for instance, there is a governmental institute whose entire role is to create new terms in Icelandic for technical terms originating in English. Neither way is more suitable than the other, it’s all simply a matter of cultural preference.

For Ibrahima, leader of the Mozilla Fulah l10n team, the choice is clear: coin new terminology. This approach presents some significant challenges. Ibrahima must decide how to best identify terms in Fulah that capture the general meaning of their English equivalents. He also must determine how to ensure that the terms are adopted within the Fulah-speaking world. While at the 2013 Silicon Valley Localization World conference, Ibrahima and I were able to chat about some specific examples of new terms he’s created while localizing Firefox into Fulah. Listen to the discussion on SoundCloud.

End of year Bugzilla housekeeping

Jeff Beatty

0

During last week. Flod had a glimpse of how messy the Mozilla Localizations product is in terms of pending bugs: 3 years old open bugs with no activity, fixed bugs still marked as NEW, etc.

Imagine a component in Bugzilla as a garden: right now Mozilla Localizations resembles an abandoned garden. With wandering zombies in it :-)

How can we improve this situation? I’m not sure if some of these features are available to all users, if not please let me know.
Follow your locale
If you’re not doing it already, start following your own locale on Bugzilla and try to keep it as clean as possible. Every time someone opens a bug in this component, you’ll receive a bugmail.

  1. Login to BugZilla with your account and go to Preferences->Component watching (https://bugzilla.mozilla.org/userprefs.cgi?tab=component_watch).
  2. Select Product “Mozilla Localizations”.
  3. Click on your locale in the “Component” section.
  4. Click Add.
Resolve bugs
If a bug is resolved, it should be marked as RESOLVED FIXED, then verified on the next build and marked as VERIFIED FIXED. Don’t leave open bugs around (remember zombies?).

Check pending bugs
You can also perform a search for open bugs in your locales and save it:

  1. Do an Advanced search on Mozilla Localizations -> Your locale  for bugs with Resolution “—”.
  2. When results are displayed you’ll see a “Remember search” button at the bottom of the page, give it a name (e.g. “Italian bugs”).
  3. From now on you’ll see a link in the footer of all Bugzilla pages to perform the same search.

Clear review and needinfo requests
If someone falgs you for a review, or set a NEEDINFO for you, you should try to clear these requests.

  1. Reviews: click on the review link in the attachment section, chose “+” if the patch is good, “-” if is not, ” ” (blank) to reset the request if you shouldn’t be reviewing the patch.
  2. Needinfos: when you reply, below the comment you’ll see a checkbox, already checked, saying “I am providing the requested information for YOURACCOUNT (clears the needinfo request).”. If you’re providing the information requested, leave it checked and comment.
When you login to Bugzilla, you should see a red circle with a white number near the title Bugzilla@Mozilla. Behind these requests there are people, so be kind to them and reply ;-)

Open letter about adding new locales to Firefox desktop

Jeff Beatty

2

First and foremost, thank you all for your interest, hard work, and time dedicated to creating an official localization of Firefox desktop. We aim to help you create high quality localizations of Firefox, because we understand that the quality of your localization work has a direct impact on people’s perception of Firefox in your region. This being the case, we ask localization teams that are interested in creating official localizations of Firefox to follow a strict process, be vocal, and make a commitment to localize each new version of Firefox every six weeks.
In the last few months, we have hesitated to add new locales to the release cycle for several reasons.  In general, we’ve not clearly emphasized the importance of key things in the process. Each of these are important to get to high quality software releases and confirm to us a prolonged interest in participating in Mozilla localization. Here’s what we hope to change in order to better support your interest in becoming an official localization: .
1) we will reach out to you to set progress goals and key milestones for your l10n work.
2) we will follow up with you when we’re unable to clearly see consistent l10n progress or we can’t gain insight into the ongoing l10n work.
3) we will encourage you to be proactive in bugzilla. Especially in the area of filing bugs and following components to help your track l10n work progress and increase visabilty.  It’s critical to interact on the localization bugs that go beyond UI translation and cover “productization.”
4) we will use the mozilla.dev.l10n and mozilla.dev.l10n.new-locales mailing lists to communicate with you more openly.
5) we will work with you to strategically build a l10n team of high quality and high value contributors.
Once again, we thank you for all of your time and effort.

 

Firefox Aurora 26 Localization Report

Jeff Beatty

0

Hello localizers!

Thank you all for your great work with Firefox 25 and 26. Here’s an outline of what is currently in Aurora this cycle (27) and what we accomplished together last cycle:

This cycle (28 October – 10 December)

Key dates include:
- Beta sign offs must be completed before 02 December.
- Aurora sign offs must be completed before 10 December.
- Firefox 26 releases 10 December.

Features:
- Approximately 145 new strings were added to Aurora desktop, 37 for Aurora mobile.
- Many of the new desktop strings cover the addition of devtools, the addition of DNT to Firefox metro, as well as crash error messages in toolkit (see previous threads on the mailing list for more details on metro and https://wiki.mozilla.org/Features/Release_Tracking#Desktop_.28Win.2C_Mac.2C_Linux.2C_Metro.29_2).
- Many of the mobile strings include improvements to context menus on about:home as well as download alerts and helper apps (see https://wiki.mozilla.org/Features/Release_Tracking#Mobile_.28Android.29_2 ).

Notes:
Please remember that sign offs are a critical piece to the cycle and mean that you approve and can vouch for the work you’re submitting for shipment.

Last cycle (17 September – 28 October)

Noteworthy accomplishments:
- 65% of all locales shipped Firefox 25 on desktop updates on time. Congratulations to everyone who signed off and shipped this last cycle! This is a 7% decrease in locale coverage between Firefox 24 and Firefox 25!
- In Firefox 25, three new locales were added to Firefox for Android: Irish (ga-IE), Ukrainian (uk), and Romanian (ro)! Congratulations! We’re continuing to add Android supported locales to this list with each new cycle in a staged approach.

Thank you to everyone for all of your dedication and hard work this last cycle!

Localization Special Interest Group (SIG) in Mozilla Reps

Jeff Beatty

2

We’re excited to announce the launch of the Localization Special Interest Group in the Mozilla Reps program! The Mozilla Reps program is a fantastic program aimed at representing Mozilla publicly. The program also helps regional Mozilla communities gather together, recruit new mozillians, and attend events where a Mozilla presence is needed.

As stated in the Mozilla Reps wiki, “Special Interest Groups (SIGs) are groups of people within the Mozilla Reps program who have a particular interest (focus) in a specific area of the Mozilla project (eg. Marketing, localization, Support, QA, add-ons etc…).

These groups are created to enable Mozilla Reps to focus on developing specific skills and work more closely with Mozilla staff responsible for those projects. Also they act as key drivers to onboard and help new volunteers to contribute to those projects they are particularly interested in.”

The Localization Special Interest Group (SIG) will aim to train Mozilla Reps on the Mozilla localization program, how and where to find skilled localizers to recruit to the Mozilla project, and connect them with their regional localization team(s). The Localization SIG will help to bridge any existing gaps between regional localization teams and regional Mozilla Reps in an effort to create sustainable relationships, networks, and teams within Mozilla Localization.

If you are interested in joining the Localization SIG, please add your name and Mozilla Reps profile link to the Members section of the main Localization SIG wiki page. We will be organizing a mailing list as well as a training days event in the near future.