zbraniecki · HackerTrans

zbraniecki

no profile record

Submissions

ICU4X 1.2: Now with Text Segmentation and More on Low-Resource Devices

blog.unicode.org

3 pointsby zbranieckihace 3 años1 comments

ICU4X 1.0 Released – New Internationalization Library from Unicode

blog.unicode.org

14 pointsby zbranieckihace 4 años0 comments

ICU4X 0.2 Released

github.com

1 pointsby zbranieckihace 5 años0 comments

ICU4X – Modular Internationalization Library in Rust

github.com

2 pointsby zbranieckihace 6 años0 comments

Join Firefox – Take your stand against an industry that’s making you the product

blog.mozilla.org

69 pointsby zbranieckihace 7 años0 comments

Async Iterators received Stage 4 and will be in ECMAScript 2018

github.com

1 pointsby zbranieckihace 9 años0 comments

Gnome 3.24 (with Night Light) Released

gnome.org

65 pointsby zbranieckihace 9 años14 comments

Rust Incremental Compilation Beta

internals.rust-lang.org

3 pointsby zbranieckihace 9 años0 comments

comments

hace 5 mesesdiscuss

Definitely the former, apologies for making it confusing.

> What is the equivalent of xgettext.pl

There is no standard one, although people build their own. The general consensus is that source strings should not be inlined into code. The closest analogy is to "style" vs "class" in HTML/CSS - the clean separation of concerns comes from the "id" being the contract.

You can read more about it here: https://github.com/projectfluent/fluent/wiki/Fluent-vs-gette...

There are attempts to "merge" those two philosophies, by extracting and "generating" slugs as ids. Examples: - https://formatjs.github.io/docs/getting-started/message-extr... - https://lingui.dev/guides/message-extraction - https://app.studyraid.com/en/read/15768/550728/setting-up-th...

I'm fairly skeptical of this approach.

> the file extension for the main catalog file `.po`

In MF1.0 world, the file format is JSON or XML. You encode id=>Message pairs. In Fluent world there is a Fluent (FTL) file format. In MF2.0 the format itself is, again, message scoped. On top of it there's a proposal by Mozilla to create MessageResource - https://github.com/w3c/i18n-discuss/blob/gh-pages/explainers... and that may feed into DOM L10n - https://github.com/mozilla/explainers/blob/main/dom-localiza...

> the __ function?

see the (1) and links to "generated ids".

> How does gender work (small example)?

MF 1.0:

``` {GENDER, select, male {He answered} female {She answered} other {They answered} } ```

Fluent: ``` user-answered = { $gender -> [male] He answered. [female] She answered. *[other] They answered. } ```

> How does layering pt_BR on pt_PT work?

MF does not prescribe fallback behavior. It also more popular to treat each locale as "complete" and fill "gaps" at build time. So at runtime you have `pt-BR` which has pt-BR strings and missing ones "completed" from `pt` (parent locale).

Fluent has a "resource manager" (simple one like this: https://github.com/projectfluent/fluent-rs/tree/main/fluent-... or more complex like Mozilla L10nRegistry), which can fallback at runtime, allowing for what we call "partial locales" which can roll out to production with gaps and the resource manager will fetch the fallback strings from the parent locale.

> What is a compelling reason to switch?

If you and your users are happy with gettext, none!

If either of those groups complain, there may be many: - https://github.com/projectfluent/fluent/wiki/Fluent-vs-gette...

Hope that helps!

hace 5 mesesdiscuss

Here is a proposal for a message resource format on top of MF2.0 - https://github.com/eemeli/message-resource-wg

hace 5 mesesdiscuss

https://messageformat.unicode.org/

Lmk if you have further questions!

hace 5 mesesdiscuss

ICU4C and ICU4J have implementations. We also have a JS polyfill and will be working on ICU4X impl this quarter.

hace 5 mesesdiscuss

Yep. Mozilla is planning an auto converter from Fluent to MF2.0 once we stabilize it.

hace 5 mesesdiscuss

We are targeting MF2.0 for inclusion in JavaEcript stdlib (ECMA-402). And later maybe with its own format into DOM for DOM L10n.

hace 5 mesesdiscuss

Correct. MF2.0 addresses all the challenges we identified during design of Fluent.

hace 5 mesesdiscuss

No, gettext scales very badly, both vertically (larger systems) and horizontally (locales with rich grammatical forms like declensions etc.)

We (authors of Fluent and collaborators on MessageFormat 2.0) wrote this explainer which you may find informative - https://github.com/projectfluent/fluent/wiki/Fluent-vs-gette...

hace 3 añosdiscuss

Hi! Thank you for your critique!

> 1) “Your knight has killed a dragon with a crossbow”

We have a proposal for dynamic references to address this problem - https://github.com/projectfluent/fluent/issues/80 - it's non-trivial but I hope we'll see it solved in Fluent and/or in MessageFormat 2.

> 2) The parser is extremely sensitive

True. It's on purpose. We wanted to start with strict and loosen, rather than the opposite.

> 3) The input files mandate a weird arrangement of new lines for even the simplest branching

Same as above.

> 4) The documentation is too Spartan to know what happens in edge cases.

We're a small team :)

> It heralds itself to be the saviour of all i18n, but it’s literally worse than the mess that came before it.

I'm sorry to hear it doesn't work for you. I'm relieved that your criticism is seems more subjective except of one missing feature that no other l10n system has as of yet. We'll keep pushing, but if you encounter a better l10n system, please let me know! We're working on Unicode MessageFormat 2.0 based on Fluent and incorporating lessons learned.

hace 3 añosdiscuss

We're happy to announce ICU4X 1.2, containing a host of new features with a focus on text engines. The new ML-powered break iterators and HarfBuzz bindings enable developers to perform text layout on many platforms and resource-constrained environments.

hace 4 añosdiscuss

Hi, just for context - this was a comparison of `unicode-normalizer` crate to ICU4C.

Since then @hsivonen from Mozilla wrote a new normalizer that recently got merged into ICU4X - https://github.com/unicode-org/icu4x/tree/main/components/no...

I don't have perf numbers yet but I suspect it to be perf comparable to ICU4C at least.

hace 5 añosdiscuss

Fair question, thanks for asking!

There have been 9 revisions of ECMA-402 since 2012. We are, in fact, following TC-39 on a yearly cadence of releases.

You can see the history of editions here: https://www.ecma-international.org/publications-and-standard...

The first edition (finished in 2012), had just NumberFormat, Collator and basic DateTimeFormat.

Since then, we added Locale, PluralRules, ListFormat, RelativeTimeFormat, DisplayNames, two new revisions of NumberFormat, two major additions to DateTimeFormat, and we're now adding Segmenter, LocaleInfo, CalendarInfo and working on MessageFormat 2.0.

Here's a very incomplete list of finished proposals that have been merged into the standard and implemented in browsers - https://github.com/tc39/proposals/blob/master/ecma402/finish...

I may or may not be involved in majority of them! :)

hace 5 añosdiscuss

You're correct. Intl formatting has two components: - Selecting the appropriate pattern for a given locale - Formatting numbers and words into the given locale (example: eastern-arabic numerals and Arabic month/week names).

You might say "I want to supply my own pattern, you just do step two for me" and technically we can provide that functionality by exposing it.

The issue is that from the API design perspective it would lead to people misusing the API misunderstanding what is going on and believing that they "internationalized their UI" which is not the case. In fact, they'd make things worse for their users than if they just displayed a date in a single locale with consistent pattern+localization because in some cases "MM/DD" and "DD/MM" are indistinguishable when expanded and that may lead to data loss, security loss, or just confusion.

I'd argue that every case where you want to supply your own pattern is a case where you should not attempt to internationalize that pattern. I also recognize that it's just my opinion.

hace 5 añosdiscuss

Hi all!

I've been working on ECMA-402 for the last 6 years on behalf of Mozilla. I'm excited to see it showcased on HN! We have an amazing, inclusive and open community of engineers, linguists and standardization exports from all of the World from largest corporations to smallest non profits and maintainers of open source little libraries.

If you'd like to see what we're working on, see https://github.com/tc39/proposals/blob/master/ecma402/README...

If you'd like to join us, please check out https://github.com/tc39/ecma402/blob/master/CONTRIBUTING.md

Our biggest challenge now is to make sure that anyone can use ECMA-402 - either with a well supported JS engine (V8, SpiderMonkey, JSShell etc.) or via library. To achieve that we're working on Rust project called ICU4X which aims to be able to back ECMA-402 in web browsers, on servers, in client solutions and offer FFI to many programming languages including to JS over WASM.

If you'd like to help us with that, there's tons of work and we're very eager to grow our community! Check out https://github.com/unicode-org/icu4x/blob/main/CONTRIBUTING.... and https://github.com/unicode-org/icu4x/tree/main/docs

If you have any questions, AMA!

hace 5 añosdiscuss

`strftime` is not an internationalization API! It's a datetime formatting API! :)

I can see an appeal for it, and I wouldn't mind `Date` or `Temporal` to have such pattern-driven formatting, but it is critical to recognize that it is not internationalization and thus doesn't belong there.

In particular, if you use `strftime` like formatting you're doing the opposite - you're hardcoding the formatting into a single pattern. It may be the right thing for your project, but it definitely is not i18n :)

hace 5 añosdiscuss

I'm one of the leaders of the ICU4X project which is a Rust implementation of ECMA-402 aiming to back client-side solutions.

We hope to eventually back SpiderMonkey (Firefox JS engine) implementation with it, but we also want to target WASM and in result expose it as a polyfill for any browser to use.

I don't know if by the time ICU4X is 1.0 IE11 will still matter, but it may be possible to compile it to asm.js and run in IE11 maybe?

hace 5 añosdiscuss

I18n parsing is indeed hell. I'm very very reluctant to try to add it to ECMA-402 for that particular reason.

Thank you for writing a user land library for it. It's a thankless task and a very important one and I think such a complex problem deserves a userland solution!

hace 5 añosdiscuss

Thanks! If you'd like to help we just merged Segmenter API into ICU4X with an intention to back the ECMA-402 Segmenter API! See https://github.com/unicode-org/icu4x/pull/717

We could definitely use some help :)

hace 5 añosdiscuss

We have been talking about this problem a bit. The tension here is that UX mocks a particular locale as "pixel perfect" and doesn't really "care" about how it'll i18n into other locales.

So they want "do the right thing" for 103 out of 104 locales, but "do what I told you" for "MY" locale.

This is a bit of a conundrum because we don't want to treat any locale as "special". To translate it to code you'd write something like:

  if (currentLocale == "THE_ONE_UX_CREATED_MOCKS_FOR") {
    let value = formatToUXProvidedPattern(data);
  } else {
    let value = data.toLocaleString(currentLocale);
  }

I don't have a great answer how to approach it since there are severe drawbacks and risks to all known potential approaches to "squeeze just one particular format that UX provided into i18n database", but I just wanted to say that we're aware that Intl API has the clash with the UX-driven-development.

hace 5 añosdiscuss

The list your linked is from 2012. In the last 9 years ECMA-402 has been heavily influenced by Mozilla's projects like Firefox OS and MozIntl APIs. :)