{"id":1284,"date":"2018-08-03T23:30:19","date_gmt":"2018-08-03T23:30:19","guid":{"rendered":"http:\/\/blog.mozilla.org\/l10n\/?p=1284"},"modified":"2018-08-06T15:45:21","modified_gmt":"2018-08-06T15:45:21","slug":"intl_pluralrules-a-rust-crate-for-handling-plural-forms-with-cldr-plural-rules","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/l10n\/2018\/08\/03\/intl_pluralrules-a-rust-crate-for-handling-plural-forms-with-cldr-plural-rules\/","title":{"rendered":"intl_pluralrules: A Rust Crate for Handling Plural Forms with CLDR Plural Rules"},"content":{"rendered":"

intl_pluralrules<\/a> is a Rust crate, built to handle pluralization<\/a>. Pluralization is the foundation for all localization<\/a> and many internationalization<\/a> APIs. With the addition of intl_pluralrules, any locale-aware date-, time- or unit-formatting (\u201c1 second\u201d vs \u201c2 seconds\u201d) and many other pluralization-dependent APIs can be added to Rust.<\/p>\n

Rust joins the family of mature languages such as C++ and Java (via ICU<\/a>), JavaScript (via ECMA 402<\/a>), and Python (via Babel<\/a>) which make writing multilingual software possible. All of those APIs use the same unified database for storing international plural rules, called Unicode CLDR<\/a>.<\/p>\n

intl_pluralrules determines the CLDR plural category for numeric input by leveraging Unicode Language Plural Rules<\/a> and plural rules from Unicode CLDR<\/a>. That category can be used to identify the correct string variant that should be used in localization.<\/p>\n

The crate is available on crates.io<\/a>, and can be used as a library in any Rust program. For example, the Rust implementation<\/a> of the Fluent Project<\/a> is the first system using intl_pluralrules for handling pluralization.<\/p>\n

Why care about plurals?<\/h1>\n

In short, numbers can change the way words appear in a string. In a simple example, the English words \u201cpage\u201d and \u201cpages\u201d are different because of pluralization. English plural forms are fairly simple; in other languages, the rules can be more complex. If software is built with one plural paradigm in mind, localizing for several languages (each with its own unique paradigm) becomes a complicated process–and unnecessarily so.<\/p>\n

\"\"<\/a><\/p>\n

Flod\u2019s blog post<\/a> on the advantages of Fluent describes the problems and potential benefits intl_pluralrules crate addresses in his section, Plural Forms. If you would like to know more about why crates like intl_pluralrules are vital to i18n and l10n in Rust, start with Flod\u2019s post.<\/p>\n

How intl_pluralrules does plurals<\/h1>\n

The intl_pluralrules crate accepts numeric input and produces the appropriate plural category for that input in a given locale. In completing this process, intl_pluralrules performs several steps outlined here:<\/p>\n

Making plural operands from numbers<\/h2>\n

Unicode\u2019s Language Plural Rules provides a defining set of characteristics that affect the plural category the number belongs to in certain languages. These characteristics are called operands<\/a>. The set comprises an absolute value, integer value, fraction value with and without trailing zeros, and the number of fraction digits with and without trailing zeros.<\/p>\n

The intl_pluralrules crate creates a set of operands<\/a> from the input and uses those operands when determining the plural category.<\/p>\n

\"\"<\/a><\/p>\n

Notice the difference between the v<\/b> value (number of visible fraction digits) for 1 and 1.0. Although 1 and 1.0 represent the same literal value (both represent a singular value), the presence of a decimal can change the plural category in some languages.<\/p>\n

For example, although in English it is unlikely to see whole count nouns measured in float style (with a trailing zero decimal), the correct plural form for 1.0 is \u201cother\u201d, not \u201cone.\u201d This means that the pages example from the previous section would read, \u201cYou have 1.0 open pages<\/i><\/b>\u201d rather than \u201cYou have 1.0 open page<\/i><\/b>.\u201d This distinction may seem strange because, as mentioned, this is an unusual use for 1.0 in English. Nonetheless, you will find that it is the proper plural form.<\/p>\n

Because Rust\u2019s float types do not preserve trailing zeros when stringified, the from<\/a> method uses Rust’s ToString<\/a> trait on its input when generating plural operands. This allows a user of the crate to send string or numeric input to the system, and, so long as it is a valid float or integer value, it will be accepted.<\/p>\n

CLDR resource parser and code generator<\/h2>\n

intl_pluralrules depends on two associated crates that reside in the same GitHub repository<\/a> and are also available on crates.io: cldr_pluralrules_parser<\/a> and make_pluralrules<\/a>.<\/p>\n

cldr_pluralrules_parser parses the plural rules from the JSON CLDR repository<\/a> and builds an AST representation of the rules. The following code snippet shows the English and Russian plural rules from CLDR.<\/p>\n

\"\"<\/a><\/p>\n

make_pluralrules generates a Rust file from that AST. The following code snippet shows the generated Rust plural logic for English.<\/p>\n

\"\"<\/a><\/p>\n

The Rust file generated by these crates is used in intl_pluralrules to determine the plural form of a number.<\/p>\n

intl_pluralrules crate<\/h2>\n

Using intl_pluralrules is a two-step process.<\/p>\n

    \n
  1. The user must create an IntlPluralRule<\/a> instance by providing a BCP 47 language tag<\/a>* and a plural type<\/a> (cardinal or ordinal) to the create method<\/a>. This will use the generated Rust file to create an IntlPluralRule<\/a> object.<\/li>\n
  2. A number needs to be passed to the select method<\/a> on the IntlPluralRule instance created in step 1.<\/li>\n<\/ol>\n

    The value returned from step 2 is the plural category.<\/p>\n

    *You can use the get_locales<\/a> method to see what languages are available in the crate<\/p>\n

    Performance and implications<\/h1>\n

    First, intl_pluralrules has landed in fluent-rs<\/a>, meaning that the Rust implementation uses the crate for handling all plural-concerned instances. Because intl_pluralrules leverages the available data from Unicode, Fluent\u2019s selection process for plural-concerned strings in any FTL file is completely automated. So long as the provided CLDR file has rules for your locale, the developer will not need to hard code plural logic into the software and localizers won\u2019t need to report a bug in order for the correct plural string to be activated.<\/p>\n

    Second, intl_pluralrules is fast. The crate is still in prerelease because, although fully functional, some optimization features are still being discussed. In spite of intl_pluralrules\u2019 WIP status regarding optimization, the system is still incredibly performant. Compared to ICU’s C PluralRules<\/a>, intl_pluralrules is approximately 20 times faster in a simple benchmark test.<\/p>\n

    Intl_pluralrules\u2019 comparative speed is due to the decision to store plural rules as compiled Rust code, rather than as CLDR syntax to be parsed at runtime. Using cldr_pluralrules_parser and make_pluralrules to generate the Rust version of the CLDR rules, the plural rules are compiled into the crate. This makes the crate slightly larger but also quicker because CLDR rules are not parsed at run time (as they are in ICU<\/a>), which is the main source of the speed disparity. As intl_pluralrules moves towards 1.0, it is expected that performance will only increase.<\/p>\n

    In the bigger picture, the release of intl_pluralrules means that the Rust ecosystem gains a higher-level internationalization and localization API, hopefully the first of several. Conversely, the internationalization and localization ecosystem gains use of this API, which leverages the performance benefits of the Rust Language.<\/p>\n

    Relevant Links:<\/p>\n