Friday, October 23, 2020

Announcing ICU4X 0.1

ICU LogoWe are thrilled to announce the first pre-release version of the ICU4X internationalization components. ICU4X aims to provide high quality internationalization components with a focus on:
  • Modularity
  • Flexible data management
  • Performance, memory, safety and size
  • Universal access from programming languages and ecosystems (FFI)
ICU4X draws from the experience of projects such as ICU4C, ICU4J, ECMA-402, CLDR, and Unicode.

Target

ICU4X is initially focusing on a subset of internationalization APIs standardized in ECMA-402 in order to cover the needs of client-side ecosystems and thin clients.

ICU4X targets a wide range of programming languages and environments, aiming to expose its APIs to languages such as Javascript, WebAssembly, Dart, C++, Python, PHP, and others.

With our focus on client-side ecosystems a lot of effort will be placed on minimizing the size, memory, and CPU utilization, and allowing for asynchronous data management.

More information on the design can be found in the project’s Announcement article.

Status

This first pre-release 0.1 version is written in Rust and introduces a small subset of APIs and scaffolding for flexible data management.

We would like to invite everyone to try it out. Take a look at the documentation and provide feedback on the API design. We’re also looking for feedback on the algorithms and data structures we use, especially from contributors with experience in Rust and ICU algorithms

More information on the release can be found in the Release Notes.

Roadmap

The next version, 0.2, will focus on validating the ability to expose ICU4X APIs to other programming environments and extending the data management system to be asynchronous.

The project is fully open source and invites all interested parties to join the effort of designing and developing a modular internationalization components system in Rust.

To learn more on how to contribute to the project, visit the CONTRIBUTE document in the project’s repository.


Over 140,000 characters are available for adoption to help the Unicode Consortium’s work on digitally disadvantaged languages

[badge]

Friday, October 9, 2020

Unicode CLDR Locale Data v38 beta available for testing

[beta image] The beta version of Unicode CLDR version 38 is now available. The data will not be changed except for showstoppers, but the LDML v38 spec can still be changed. The final release of v38 is planned for October 28, 2020. If you find any problems, please file a ticket.

Unicode CLDR provides an update to the key building blocks for software supporting the world's languages. CLDR data is used by all major software systems (including all mobile phones) for their software internationalization and localization, adapting software to the conventions of different languages.

CLDR v38 includes:
  • Enhancements to existing locale data: adding support for units of measurement in inflected languages (phase 1), adding annotations (names and search keywords) for Unicode symbols that are non-emoji (~400), and annotations for  Emoji v13.1. 
  • Survey Tool upgrades: substantial performance improvements, plus structured forum entries to improve coordination among translators.
LDML v38 includes:
  • To make the canonicalization of locale identifiers clear and unambiguous, provided major restructuring of the specification for it. (This was done in concert with fixes to the alias data to work better with the specification.)
  • To support inflected units of measurement:
    • minimalPairs adds new elements
      caseMinimalPairs and genderMinimalPairs
    • unit adds a new element gender
    • grammaticalData adds new elements
      grammaticalDerivations, deriveCompound, and deriveComponent
    • unitPattern adds a new attribute case
    • grammaticalCase, grammaticalGender, grammaticalDefiniteness add a new attribute scope
    • compoundUnitPattern1 adds new attributes case and gender
    • compoundUnitPattern adds a new attribute case
  • To allow for overriding dictionary-based segmentation breaks, added the Unicode Dictionary Break Exclusion Identifier, with the new key “dx”.
  • For picking the correct units of measurement for locales, defined the userPreferences skeleton more precisely.
  • For accurate plural categories in compact numbers, added the 'c' operand to plural rules to provide formatting for languages such as French.
See additional details in the draft CLDR v38 Release note.

The overall changes to the data items were:

Added Deleted Changed Total
155,131 33,805 45,895 2,175,821


Over 140,000 characters are available for adoption to help the Unicode Consortium’s work on digitally disadvantaged languages

[badge]