# Localization
Translation catalogs in [Mozilla Fluent](https://projectfluent.org/), a locale
negotiated per request from `Accept-Language` and from overrides the
application opts into, and an active locale a handler, a view or an Inertia
page can read.
A locale tag is matched against the catalogs the application registered. It
never becomes a filesystem path, because there is no filesystem access in the
subsystem at all.
## Turning it on
The feature is `i18n`, and it is off everywhere:
```toml
arcature = { version = "0.1", features = ["i18n"] }
```
| framework `default` | off |
| framework `fullstack` | off |
| generated application | off, and `arc new` scaffolds nothing for it |
`i18n = ["dep:fluent-bundle", "dep:unic-langid"]`, both with
`default-features = false`. `fluent-bundle` is the reference Rust
implementation of Fluent — the `.ftl` parser, the formatter, and the CLDR
plural-rule selection. `unic-langid` is the BCP-47 language-identifier type
that API is written in; it is a direct dependency because `fluent-bundle` does
not re-export it and a locale has to be parsed before a bundle can be built.
The features left off are `all-benchmarks` on `fluent-bundle`, and `macros`,
`likelysubtags` and `serde` on `unic-langid`: a benchmark harness, a
proc-macro, a data table and a derive, none of which this crate uses.
Nothing enables `i18n` for you, and nothing in the framework installs the
locale layer on your behalf. There is no `Application::i18n(..)` builder
method. Wiring is described under [Installing the
layer](#installing-the-layer).
Where the names live:
| `arcature::` (crate root) | `Catalog`, `Catalogs`, `I18nError`, `Locale`, `LocaleId`, `LocaleLayer` |
| `arcature::i18n::` | those, plus `ArgValue`, `LocaleMiddleware`, `LocaleNegotiator`, `LocaleRejection`, `LocaleSource`, `TranslationArgs` |
| `arcature::prelude::*` | **nothing** — the prelude carries no `i18n` name |
A handler that takes a `Locale` writes `use arcature::i18n::Locale;` even with
the prelude imported.
## Why Fluent and not a map
The obvious implementation of translation is `HashMap<String, String>` keyed by
locale. It is wrong for every language whose grammar is not English's, in three
separate ways, and this feature exists to pick the engine that gets all three
right.
**Plurals.** English has two categories, so a map with a `_one` key and an
`_other` key looks complete. It is not a property of the message; it is a
property of the language, and CLDR gives Polish four categories, Arabic six and
Japanese one. The selection cannot be an `if n == 1` in the calling code,
because the calling code does not know which language it is rendering, and the
rule for that language is a table rather than an arithmetic expression a
developer can guess.
**Agreement.** "the file was deleted" has a gendered participle in French and
in Russian. The correct string depends on a property of an argument, not only
on the key, so no amount of keys fixes it.
**Numbers and dates.** `1,234.5` is `1 234,5` in French and `1.234,5` in
German. A value formatted before it reaches the map is formatted in the
server's locale rather than the reader's.
Fluent puts the decision inside the catalog, where the translator can see it
and change it, instead of inside the calling code, where they cannot. Adding a
language with four plural categories is an edit to one `.ftl` file and to
nothing else:
```rust
use arcature::i18n::{Catalog, LocaleId, TranslationArgs};
let catalog = Catalog::parse(
LocaleId::parse("pl").unwrap(),
r"files = { $count ->
[one] plik
[few] pliki
[many] plikow
*[other] pliku
}",
)
.unwrap()
.isolating(false);
.translate("files", &TranslationArgs::new().with("count", n))
.unwrap()
};
assert_eq!(of(1), "plik");
assert_eq!(of(2), "pliki");
assert_eq!(of(5), "plikow");
assert_eq!(of(22), "pliki");
```
The calling code passed an `i64` three times and knew nothing about Polish.
The price is a runtime parser, which the [Views](views.md) feature rejected on
purpose. That is a real tension and it gets its own answer under
[Security](#security).
## Writing a catalog
`Catalog::parse(locale, ftl)` turns `.ftl` source into a catalog. In an
application the source is `include_str!`, so the bytes are in the binary and
the parse is a startup cost:
```rust,ignore
use arcature::i18n::{Catalog, LocaleId};
let english = Catalog::parse(
LocaleId::parse("en")?,
include_str!("../locales/en.ftl"),
)?;
```
`locales/en.ftl` is an ordinary file in the repository. Nothing at runtime
reads it; nothing at runtime can be told to read a different one.
| `Catalog::parse(locale, ftl)` | parse one source into a new catalog |
| `catalog.with_source(ftl)` | fold another source into the same catalog |
| `catalog.isolating(bool)` | Unicode bidi isolation of placeables; **on by default** |
| `catalog.locale()` | the `&LocaleId` this catalog is for |
| `catalog.has(key)` | whether the catalog defines a message under `key` |
| `catalog.message(key)` | format a message that takes no arguments |
| `catalog.translate(key, &args)` | format a message with arguments |
| `catalog.attribute(key, attr, &args)` | format one attribute of a message |
`with_source` is how a large application splits its messages by area without
splitting the locale. It refuses to redefine a message or a term the catalog
already has — a silent overwrite would make a key's meaning depend on the
order the files happened to be added in:
```rust
use arcature::i18n::{Catalog, LocaleId};
let catalog = Catalog::parse(LocaleId::parse("en").unwrap(), "a = A").unwrap();
assert!(catalog.with_source("a = B").is_err());
```
Attributes keep the strings of one UI element together so a translator sees
them as a unit:
```rust
use arcature::i18n::{Catalog, LocaleId, TranslationArgs};
let catalog = Catalog::parse(
LocaleId::parse("en").unwrap(),
"search = Search\n .placeholder = Search the archive",
)
.unwrap();
assert_eq!(catalog.message("search").unwrap(), "Search");
assert_eq!(
catalog
.attribute("search", "placeholder", &TranslationArgs::new())
.unwrap(),
"Search the archive"
);
```
### Isolation marks
`isolating` is on by default, and it should stay on in anything a person reads.
Fluent wraps every placeable in `U+2068` and `U+2069`, the Unicode isolation
marks, so an Arabic name interpolated into an English sentence does not drag
the punctuation around it to the other side of the line. They are invisible in
a browser and visible in a byte-for-byte assertion, which is the only reason
`isolating(false)` exists: tests, and non-display sinks such as a log line.
Most runnable examples in this chapter that compare a formatted string call
it. Three do not, because the strings they assert on carry no placeable and so
no isolating marks appear in them.
### Arguments
`TranslationArgs` is a builder of named values:
```rust
use arcature::i18n::{Catalog, LocaleId, TranslationArgs};
let catalog = Catalog::parse(
LocaleId::parse("en").unwrap(),
"invoice = { $name } owes { $amount }",
)
.unwrap()
.isolating(false);
let args = TranslationArgs::new().with("name", "Ada").with("amount", 12.5);
assert_eq!(catalog.translate("invoice", &args).unwrap(), "Ada owes 12.5");
```
`with` takes anything that converts into an `ArgValue`, which has three
variants and these `From` impls and no others:
| `Text(String)` | `String`, `&str` | interpolated as-is; escaping is the view layer's job |
| `Integer(i64)` | `i64`, `i32`, `u32`, `usize` | `usize` **saturates** at `i64::MAX` rather than wrapping |
| `Number(f64)` | `f64` | |
There is no `u64`, no `f32`, no `u8`/`u16`/`i8`/`i16` and no `bool`. Convert at
the call site.
Integer and float are separate cases because CLDR treats them separately: in
several languages `1` and `1.0` fall into different plural categories, and
collapsing both into an `f64` would quietly pick the wrong one.
Setting the same name twice keeps the later value. Arguments keep insertion
order, and `TranslationArgs` is a `Vec` behind the scenes rather than a map,
because a message has a handful of placeables and a linear scan over three
entries beats hashing three keys.
`TranslationArgs` is deliberately not `fluent_bundle::FluentArgs`. `FluentArgs`
in a public signature would make `fluent-bundle`'s version part of Arcature's
public API, and a `0.16` to `0.17` bump upstream — routine for a crate at
`0.x` — would become a breaking change here.
## Registering locales
`Catalogs` is the registry, and it is also the whitelist: it is the only answer
in the framework to "is this a locale this application has?".
```rust
use arcature::i18n::{Catalog, Catalogs, LocaleId};
let en = LocaleId::parse("en").unwrap();
let fr = LocaleId::parse("fr").unwrap();
let catalogs = Catalogs::new(
Catalog::parse(en.clone(), "greeting = Hello\nfarewell = Goodbye").unwrap(),
)
.with(Catalog::parse(fr.clone(), "greeting = Bonjour").unwrap());
assert_eq!(catalogs.default_locale(), &en);
assert_eq!(catalogs.message(&fr, "greeting").unwrap(), "Bonjour");
assert!(!catalogs.contains(&LocaleId::parse("de").unwrap()));
```
| `Catalogs::new(catalog)` | start the registry from the catalog that is also its default |
| `catalogs.with(catalog)` | register another locale; the same locale twice keeps the later catalog |
| `catalogs.default_locale()` | the `&LocaleId` used when nothing better is registered |
| `catalogs.default_catalog()` | the default locale's catalog, which always exists |
| `catalogs.contains(&locale)` | the whitelist test |
| `catalogs.catalog(&locale)` | `Option<&Catalog>` |
| `catalogs.locales()` | every registered locale, in canonical-tag order |
| `catalogs.message(&locale, key)` | format a no-argument message, with fallback |
| `catalogs.translate(&locale, key, &args)` | format a message, with fallback |
`new` takes a `Catalog` and not a `LocaleId` on purpose. A registry whose
default locale has no catalog is a configuration that fails at the first
request in the worst language, and this signature makes it unspellable.
Nothing removes a catalog. The set is fixed once the registry is built, from
values the application's own code supplied.
`Catalogs` is `Clone` and the catalogs sit behind an `Arc`, so a copy per
request costs a refcount bump. `Catalog` itself is not `Clone`; build it once
and hand it to the registry.
### Message fallback
There are two levels of fallback and they are separate on purpose.
*Locale* fallback happens before a catalog is chosen and belongs to
negotiation: an unregistered locale becomes the default one.
*Message* fallback happens inside `Catalogs::translate`: a key the chosen
catalog does not have is looked up in the default catalog before the call
fails.
```rust
use arcature::i18n::{Catalog, Catalogs, LocaleId};
let fr = LocaleId::parse("fr").unwrap();
let catalogs = Catalogs::new(
Catalog::parse(
LocaleId::parse("en").unwrap(),
"greeting = Hello\nfarewell = Goodbye",
)
.unwrap(),
)
.with(Catalog::parse(fr.clone(), "greeting = Bonjour").unwrap());
// `farewell` was never translated. The page renders in English rather than
// failing.
assert_eq!(catalogs.message(&fr, "farewell").unwrap(), "Goodbye");
```
That is what makes a partially translated locale usable: a new string ships in
`en`, and a `fr` page shows the English sentence rather than a `500` or an
empty span. It is per key, so it cannot hide a missing catalog — a locale with
no catalog at all was never selectable.
Two things the fallback deliberately does not do:
* **A formatting failure is not retried against the default catalog.** The
message was found and the caller's arguments did not fit it; the same
arguments will not fit the English one either.
* **`Catalogs` has no `attribute` method.** Attributes are reachable only
through `Catalog::attribute`, which is one catalog, so an attribute a locale
has not translated is `I18nError::Missing` rather than the default catalog's
text. See [What this does not do](#what-this-does-not-do).
A key that is missing everywhere reports the locale that was *asked for*, not
the default one, so the error names the language the reader was in.
## Locale identifiers
`LocaleId` is a validated, canonical BCP-47 language identifier, and it is the
only locale type the API accepts.
```rust
use arcature::i18n::LocaleId;
// Canonical casing is applied on the way in.
let locale = LocaleId::parse("en-us").unwrap();
assert_eq!(locale.as_str(), "en-US");
assert_eq!(locale, LocaleId::parse("EN-US").unwrap());
assert_eq!(locale.language(), "en");
assert!(LocaleId::parse("../../etc/passwd").is_err());
```
`parse` is the only constructor and it is fallible. Validation runs in two
passes: a cheap allocation-free shape check, then the real parser on a string
already known to be short and alphanumeric.
| maximum length | **35 bytes**, checked before parsing |
| subtag length | 1--8 bytes |
| byte set | ASCII alphanumerics, `-` as the separator only |
| subtag order | language, script, region, variants |
| casing | canonicalized, so `zh-hant-hk` and `ZH-HANT-HK` are one locale |
Everything else is refused: `..`, `/`, `\`, a NUL byte, a newline, a space, an
underscore, an empty subtag, a percent-encoded sequence, a bidi override
character, a 4 KB string. `LocaleId::parse` is the one place that check has to
be written, which is the point of the newtype — a `String` carries no history,
so the check would otherwise have to be repeated at every use and the one call
site that forgot would be the bug.
`LocaleId` implements `Display`, `AsRef<str>`, `FromStr`, `Serialize`, `Clone`,
`Ord` and `Hash`. It does **not** implement `Deserialize`: a struct with a
`LocaleId` field cannot `#[derive(Deserialize)]`. Take a `String` and call
`parse` on it.
## Negotiating a request locale
`LocaleNegotiator` turns what a request proposed into a registered locale. The
whole design is one sentence: a request proposes locales, `Catalogs` decides
which of them exist, and anything unproposed, unparseable or unregistered
becomes the default.
```rust
use arcature::i18n::{Catalog, Catalogs, LocaleId, LocaleNegotiator, LocaleSource};
let catalogs = Catalogs::new(
Catalog::parse(LocaleId::parse("en").unwrap(), "hi = Hello").unwrap(),
)
.with(Catalog::parse(LocaleId::parse("fr").unwrap(), "hi = Bonjour").unwrap());
let negotiator = LocaleNegotiator::new(catalogs)
.query_parameter("lang")
.session_key("locale");
// A browser configured for French, with no explicit choice on record.
let locale = negotiator.resolve(None, None, Some("fr-CA,fr;q=0.9,en;q=0.4"));
assert_eq!(locale.id().as_str(), "fr");
assert_eq!(locale.source(), LocaleSource::Header);
// An explicit `?lang=` beats both the session and the header.
let locale = negotiator.resolve(Some("en"), Some("fr"), Some("fr"));
assert_eq!(locale.id().as_str(), "en");
assert_eq!(locale.source(), LocaleSource::Url);
// A hostile tag is not a candidate.
let locale = negotiator.resolve(Some("../../etc/passwd"), None, None);
assert_eq!(locale.source(), LocaleSource::Default);
```
### Precedence
`LocaleSource`, most specific first:
| `LocaleSource::Url` | the query parameter named by `.query_parameter(..)` | **off** |
| `LocaleSource::Session` | the session key named by `.session_key(..)` | **off** |
| `LocaleSource::Header` | the request's `Accept-Language` | always read |
| `LocaleSource::Default` | nothing the request offered was registered | — |
Both overrides start off, and an override that was never configured is not an
override: an application that has not opted into `?lang=` cannot have a user's
locale changed by a link somebody emailed them. A URL parameter puts the locale
into every link a page emits and into every log line; a session key is a write
to storage the application owns. Neither should appear because a framework
assumed a name for it.
`LocaleSource` is `#[non_exhaustive]`. Match with a catch-all arm.
Two properties of the overrides worth knowing before you turn them on:
* **The query value is taken verbatim, without percent-decoding.** A
well-formed locale tag is `[A-Za-z0-9-]` and needs none, so a percent-encoded
tag fails validation and falls through to the next source. That is one
decoder fewer between a request and a lookup. A repeated `?lang=` resolves to
the **last** occurrence, matching what `serde_urlencoded` does, so a
parameter smuggled in ahead of the real one does not take precedence over it.
* **The session is read and never written.** Persisting a choice is a decision
with a cookie and a lifetime attached, and it belongs to the handler that
offers the language switcher. Reading it needs the `auth` feature, which is
what brings `tower-sessions`; without `auth` the key is accepted and never
consulted, so enabling `auth` later does not change a call site. The value
goes through `LocaleId::parse` like everything else — "we wrote it, so it is
fine" is how a validated field stops being validated.
### Matching a proposed tag
Every proposed tag, from any source, goes through exactly two steps:
1. `LocaleId::parse`, which refuses anything that is not a canonical BCP-47
identifier of at most 35 bytes;
2. a lookup in `Catalogs`, which is an in-memory `BTreeMap` whose keys came
from the application's own source at startup.
A tag that fails either is discarded and the next candidate is tried. On an
exact miss there is one further step: the first registered locale whose
*language subtag* equals the candidate's language subtag. That matching runs on
an already-validated identifier, never on a prefix of the raw string — a
prefix match on raw bytes would make `en/../../etc` match `en`.
The consequence is a fallback in both directions:
| `en`, `fr`, `pt-BR` | `fr-CA` | `fr` — a region falls back to its language |
| `en`, `fr`, `pt-BR` | `pt` | `pt-BR` — a language falls back to a registered region |
| `en`, `fr`, `pt-BR` | `de-DE` | `en` — the default |
| `en`, `fr`, `pt-BR` | `fr-`, `frx`, `fr..` | `en` — not a prefix match |
Candidates are tried in order, and each candidate is resolved exact-first then
by language. There is no pass that prefers an exact match somewhere later in
the list over a language match earlier in it: with `fr` and `pt-BR` registered,
`Accept-Language: pt-PT, fr` selects `pt-BR`, because `pt-PT` resolves by
language before `fr` is ever reached. When several registered locales share a
language subtag the first in canonical-tag order wins, so `pt` with both
`pt-BR` and `pt-PT` registered selects `pt-BR`.
### Accept-Language
The header is parsed into weighted candidates, best first.
| absent `q` | 1.0, per RFC 9110 |
| `q=0` | "not this one" — the entry is dropped, not ranked last |
| malformed `q` | treated as absent rather than as zero |
| `q` outside `0.0..=1.0` | treated as absent |
| equal weights | stable, so the client's order is kept |
| `*` | dropped; "anything" is what falling through to the default already does |
| parameters after `;` that are not `q` | discarded |
| whitespace and casing | tolerated |
| bytes read | the **first 512**; the rest of the header is ignored |
| entries read | the **first 16**; the rest are ignored |
The two bounds are why `Accept-Language: en;q=0.1,` repeated ten thousand times
followed by `fr` selects `en`: the `fr` at the far end is never seen. A real
browser sends well under 100 bytes and at most a dozen entries, so the bound
costs nothing real and removes the reason to walk a megabyte of padding.
Weights are compared as thousandths (`q=0.8` is `800`), so the sort is over
integers rather than over a partial order on `f32`.
A hostile entry does not take the rest of the header down with it:
`../../etc/passwd;q=1.0,\0;q=0.95,fr;q=0.9` selects `fr`.
### Installing the layer
`LocaleLayer` negotiates once per request, puts the `Locale` in the request's
extensions, and annotates the response.
```rust,ignore
use arcature::i18n::{LocaleLayer, LocaleNegotiator};
Application::new()
.routes(routes())
.layer(LocaleLayer::new(
LocaleNegotiator::new(catalogs).query_parameter("lang"),
))
.build()
```
On the way out it sets two headers:
| `Content-Language` | the selected tag — **only if the handler did not already set one** |
| `Vary` | `Accept-Language` appended to whatever was there, never duplicated; a `Vary: *` is left alone |
The `Vary` is not decoration. Without it a shared cache stores one
representation per URL and serves the French page to the next English reader,
which is a correctness bug at best and a privacy one as soon as a page contains
anything about the person who requested it. An existing
`Vary: accept-language` in any casing is recognised and left as it is.
An `Accept-Language` header whose bytes are not visible ASCII is treated as
absent rather than as an error: `HeaderValue::to_str` refuses it, and the
request is served in the default locale with a `200`.
**Where you install the layer decides what sees the locale.** Layers added with
`Application::layer(..)` are stage 21 of the request pipeline — the innermost
stage, inside the session (stage 16) and inside Inertia (stage 18);
`src/application/pipeline.rs` has the full table. That is the right side of the
session, so the session override works. It is the wrong side of Inertia for one
path only:
| the `Locale` extractor | yes — an extractor runs after every layer on the route |
| the `Inertia` extractor and `inertia.render(..)` | yes, for the same reason |
| a handler returning `Page<T>`, rendered by `InertiaLayer` after the fact | **no** |
`InertiaLayer` reads the locale out of the extensions when *it* runs, which is
before a user layer. A deferred `Page<T>` render therefore loses its `locale`
prop and nothing else. To fix it, install `LocaleLayer` on a `Router` outside
`InertiaLayer` — with `Router::layer` the last layer applied is the outermost,
so `LocaleLayer` is the later call:
```rust,ignore
let app: Router = Router::new()
.route("/users", get(index))
.layer(InertiaLayer::new(config))
.layer(LocaleLayer::new(negotiator));
```
## Reaching the locale from a handler
`Locale` is an extractor. It reads what the layer put in the extensions; it
does not negotiate:
```rust,ignore
use arcature::i18n::Locale;
use arcature::prelude::*;
async fn greet(locale: Locale) -> Result<Response> {
Ok(text(StatusCode::OK, locale.message("greeting")?))
}
```
| `locale.id()` | `&LocaleId`, the active tag |
| `locale.source()` | `LocaleSource` — worth reading in a language switcher |
| `locale.is_default()` | whether the locale was fallen back to rather than asked for |
| `locale.catalogs()` | `&Catalogs`, every locale the application registered |
| `locale.catalog()` | `&Catalog` for this locale; always present |
| `locale.message(key)` | `Result<String, I18nError>` |
| `locale.translate(key, &args)` | `Result<String, I18nError>` |
Cloning is cheap: the tag is an `Arc<str>` and the catalogs are behind an
`Arc`.
The extractor deliberately does not negotiate a locale of its own. Doing that
would need a `Catalogs`, which only the layer has, and it would make a route
that forgot the layer answer in a language nobody negotiated instead of saying
the wiring is missing. On a route with no `LocaleLayer` the extractor produces
`I18nError::NotNegotiated`, which becomes a `500`. What the client sees is
covered under [Error handling](#error-handling); the short version is that it
does not name the layer.
There is no `OptionalFromRequestParts` impl, so `Option<Locale>` is not an
extractor. A route either has the layer or does not.
## From a view
With `views` on, translation happens in the template: give the template struct
a `Locale` field and call it.
```rust,ignore
#[derive(Template)]
#[template(
source = "<h1>{{ locale.message(\"hi\").unwrap_or_default() }}</h1>",
ext = "html",
askama = arcature::askama
)]
struct Greeting {
locale: arcature::i18n::Locale,
}
let response = view(Greeting { locale: locale.clone() })
.in_locale(&locale)
.into_response();
```
Two things to notice.
**There is no `{{ t("key") }}` filter and there will not be one.** Adding one
would mean a lookup the compiler cannot check, which is the opposite of the
reason the view layer compiles its templates at all. The cost is visible in the
example: a template cannot use `?`, so a failed lookup needs
`unwrap_or_default()` or a field the handler resolved before rendering.
**`in_locale` is a separate call, and `Content-Language` is absent without
it.** The framework does not infer the header in either direction. A compiled
template carries no language — askama resolved it to `write!` calls — and the
locale `LocaleLayer` negotiated is what the request *asked* for, which is not
the same claim as what the bytes in this response are actually in. A handler
that renders a French template says so; one that renders a template it did not
translate says nothing, which is better than an untrue header.
`LocaleLayer` will not fill the gap either: it sets `Content-Language` only
when the response does not already carry one, and a `View` that never called
`in_locale` carries none, so the layer's value is used. The two agree exactly
when the template really was rendered in the negotiated locale, which is the
claim `in_locale` exists to let a handler make explicitly.
## From Inertia props
With `inertia` on, the renderer publishes the negotiated locale as a `locale`
prop:
```json
{
"id": "fr",
"source": "header",
"available": ["en", "fr"]
}
```
`source` is `"url"`, `"session"`, `"header"` or `"default"`. `available` is
every registered locale in canonical-tag order, which is the list a language
switcher needs. It is an object rather than a bare string so it can grow
without changing a shape a client already destructures.
The `Inertia` extractor exposes the same value to a handler that wants to
translate something itself: `inertia.locale()` returns
`Option<&arcature::i18n::Locale>`.
Three rules, and each is a rule about *not* acting:
| the application already shares a `locale` prop | its prop wins, untouched |
| a partial reload that did not name `locale` in `X-Inertia-Partial-Data` | no `locale` prop |
| a partial reload that did name it | the prop is sent |
| no `LocaleLayer`, or the layer ran too late | no `locale` prop, and the page renders as it did before |
Overwriting an application's own `locale` prop would break a working page on a
feature flag; adding an unrequested prop to a partial response is exactly the
payload growth partial reloads exist to avoid.
The prop is inserted after prop resolution and is not a field of any `#[page]`
struct, so it is outside the Client Exposure Firewall's schema and `arc
typegen` does not know about it. **A generated page type will not have a
`locale` field.** Declare it in the TypeScript by hand, or add your own
`locale` prop to the page struct, which then wins under the first rule above.
## Security
Every claim here is a property of `src/i18n/`, checked against the source
rather than assumed from a type's name.
### There is no filesystem access
`src/i18n/` performs no filesystem access at all. A search of the module for
`std::fs`, `std::io`, `tokio::fs`, `File`, `Path`, `PathBuf`, `read_to_string`
and `Command` returns one line, and it is a comment discussing the absence. The
whole module is `args.rs`, `catalog.rs`, `error.rs`, `locale.rs`,
`negotiate.rs` and `mod.rs`, and none of them opens anything.
Catalogs are values the application constructs and hands over. The registry is
an in-memory `BTreeMap` built at startup. Lookup is a map lookup against
`Catalogs::contains`, a whitelist whose entries came from the application's own
source.
This matters because the classic way to lose is to turn a locale tag into
`locales/{tag}.ftl` and open it, at which point `../../etc/passwd` and
`..\..\..\windows\win.ini` are one request away. There is no filesystem path
for a hostile tag to traverse because there is no filesystem access to
traverse it with. The property is structural, not defended.
### A hostile or unregistered locale selects the default
That is the belt; `LocaleId` is the braces. It is the only locale type the API
accepts, its only constructor validates, and a request's raw string cannot be
passed where a locale is expected without going through it.
Each of these is something a request can carry, and none of them parses, so
none of them is ever a candidate — from `?lang=`, from the session, or from
`Accept-Language`:
```text
../../etc/passwd ..\..\..\windows\win.ini /etc/passwd
C:\Windows\System32 en/../../etc/passwd %2e%2e%2f%2e%2e%2f
....//....//etc/passwd en\0 \0/etc/passwd
fr\nSet-Cookie: stolen fr\r\nX-Injected: 1 $(cat /etc/passwd)
`id` {{7*7}} <script>alert(1)</script>
en\u{202e} \u{feff}en "e" x 64000
```
In every case the outcome is the application's default locale with
`LocaleSource::Default` — a value that was never derived from the input at
all. A well-formed but unregistered tag such as `de-DE` takes the same path.
`src/i18n/negotiate.rs` pins this in a test that walks the list above through
all three sources.
One entry deserves a note, because it looks like an exception and is not.
`Accept-Language: en; rm -rf /` selects `en`, with `LocaleSource::Header`. That
is not a hostile tag being accepted: `;` starts a parameter list, so the
language range is `en` and the rest is a parameter that is not a `q` and is
discarded, which is what RFC 9110 says the field means. The `en` still goes
through `LocaleId::parse` and the whitelist like any other candidate, and
nothing after the `;` survives into the result. The same bytes offered as a
whole tag through `?lang=` are refused.
### The Fluent parser runs over developer-authored catalogs
The view layer chose askama specifically so that no template parser runs inside
the request path. This module adds a runtime parser. The tension is real and
the answer is that the two parsers eat different food.
**A catalog is developer-authored and lives in the repository.** The `.ftl`
text passed to `Catalog::parse` is a file a translator wrote and a reviewer
merged. It does not arrive over the network and it is not selected by anything
a request controls. In the intended use it is `include_str!`, so the bytes are
in the binary and the parse is a startup cost.
**A request supplies arguments, not messages.** Values from a request reach
Fluent as `ArgValue`s — a string, an integer, a float — and Fluent
interpolates them. It does not evaluate them: there is no path by which a
`$name` of `{ $other }` becomes a placeable, because the message's pattern was
fixed when the catalog was parsed and an argument is substituted into that
pattern rather than re-parsed with it. Fluent has no property lookup on host
objects, no filesystem access and no `eval`, and a catalog here can invoke no
functions at all: `NUMBER` and `DATETIME` exist in Fluent but require
`FluentBundle::add_builtins`, which `Catalog::parse` never calls. The machinery a template-injection
payload needs is not there to reach.
That leaves one rule this module holds itself to, and it is the thing to check
in review: **never call `Catalog::parse` on bytes that came from a request.** A
feature that let an administrator upload a `.ftl` file, or that read a catalog
out of a database row, would put attacker-influenced text into the parser and
would need its own analysis. Nothing here does that, and nothing here offers a
way to.
### Errors quote the catalog, never the request
Two rules shape what `I18nError` is allowed to carry.
**A rejected locale tag is never quoted back.** `LocaleRejection` says what was
wrong and never what was sent, in three coarse variants — `Empty`, `TooLong`,
`NotWellFormed`. The string that failed validation is the one place in this
subsystem where a request's bytes arrive unexamined, and writing it into an
error's `Display` puts it one `tracing::warn!` away from a log line. A log line
is a text format with no escaping: a tag containing `\n` writes a second entry,
and a tag containing a terminal escape sequence is read by whoever `cat`s the
file. The three variants are enough to debug a developer's own typo.
**A message key is not a secret, and a translated string may be.** A key is a
constant in the application's source, so naming one in an error is safe and
useful. The formatted value is not: it can carry whatever the caller
interpolated. `I18nError::Format` therefore reports Fluent's diagnostics and
never the partially formatted output — Fluent's recovery for an unresolved
placeable is to emit the placeable's own source text, so what it hands back on
the error path is a string with `{ $count }` in it, and a caller holding a
`String` will put it on a page.
## Error handling
`I18nError` has five variants:
| `InvalidLocale(LocaleRejection)` | a string was refused as a locale tag; the string is not carried |
| `Parse { locale, errors }` | an `.ftl` source did not parse, or its messages collided with ones already in the catalog |
| `Missing { locale, key }` | neither the locale's catalog nor the default one has the key |
| `Format { locale, key, errors }` | the message exists but could not be formatted |
| `NotNegotiated` | a handler asked for `Locale` on a route without `LocaleLayer` |
`Parse` is a developer error by definition: the source is a file in the
repository, so a catalog that fails here fails on the first request after a
deploy, for everyone, identically. `NotNegotiated` is wiring, not input: it is
the same for every request that reaches that route and it is fixed in one line
of the router.
`From<I18nError> for arcature::Error` produces `Error::Other("translation
failed")`, which answers **status 500, code `internal_error`**, for all five
variants. The key, the locale and Fluent's diagnostics are dropped, because
`Error`'s `IntoResponse` writes its `Display` text into the `detail` field of
the problem document outside production — so anything left in it is one
`APP_ENV` away from the wire, and a message key is a fragment of the
application's source tree.
The detail goes to `tracing::error!` instead. **`tracing` arrives with
`observe`, and `i18n` does not imply it.** In a build with `i18n` and without
`observe` the diagnostics are discarded with nothing recorded anywhere: the
client gets its uninformative `500` and the operator gets silence. This is the
same trap [Views](views.md#render-failures) documents, and the same advice
applies — if you enable `i18n`, enable `observe`.
Concretely, a missing `LocaleLayer` on a route produces a `500` whose body says
`translation failed` and mentions neither the layer nor negotiation. There is a
test asserting exactly that. Without `observe`, nothing anywhere says what went
wrong.
## `unsafe` in the dependency tree
`arcature` is `#![forbid(unsafe_code)]`. Its dependencies are not, and two
facts about this feature belong in the open.
**`fluent-bundle` pulls in `self_cell`, which contains `unsafe`.** `self_cell`
is how `FluentResource` holds a `String` of `.ftl` source together with an AST
that borrows from it — a self-referential struct, which safe Rust cannot
express, so the crate builds one with a small amount of `unsafe` and a
well-known soundness argument. It is not incidental: it is the reason parsing a
catalog does not copy every string out of the source. Enabling `i18n` accepts
that.
The rest of the subtree is pure Rust with no C, no network and no filesystem
access:
| `fluent-syntax` | the `.ftl` parser |
| `fluent-langneg` | language negotiation primitives |
| `intl-memoizer` | caches per-locale formatters |
| `intl_pluralrules` | the CLDR plural-category tables |
| `unic-langid`, `unic-langid-impl` | the BCP-47 identifier type |
| `type-map`, `rustc-hash`, `smallvec` | containers |
| `self_cell` | **contains `unsafe`** |
**The `cargo geiger` baseline does not change, and that is not a claim that
nothing was added.** `baselines/unsafe-baseline.<host-target>.txt` is recorded by `just
geiger`, which runs `cargo geiger --all-targets` over the *default* feature
set. `i18n` is not in `default`, so `self_cell` is outside the graph the
baseline measures and the file is byte-identical. A reader who expected the
number to move should know why it did not, rather than conclude the dependency
is free: an application that turns `i18n` on takes on `self_cell`'s `unsafe`,
and that is not visible in the recorded numbers.
## What this does not do
Collected, because a chapter that lists only what works is useless to somebody
deciding whether to depend on this.
* **No catalog loading of any kind.** No directory scan, no `locales/` convention,
no reload. `Catalog::parse` takes a `&str` and the application decides where
it came from. This is the security property, not an omission to be fixed.
* **No hot reload.** A catalog compiled in with `include_str!` changes when the
crate is rebuilt. `arc dev` watches `.rs`, `Cargo.toml`, `Cargo.lock` and
`.env*`, so saving an `.ftl` file triggers nothing — but the `include_str!`
makes rustc track it, so `cargo build` or touching any `.rs` picks it up.
* **No fallback for attributes.** `Catalogs` has `message` and `translate` and
no `attribute`. An attribute is only reachable through `Catalog::attribute`,
on one catalog, so an untranslated `.placeholder` is `I18nError::Missing`
rather than the default catalog's text. Message-level fallback does not
extend to it.
* **No session write.** The negotiator reads the session key and never sets it.
A language switcher that should persist a choice writes the session itself.
* **No cookie source.** `Accept-Language`, a query parameter and a session
entry are the three sources. A `locale` cookie is not one of them; read it in
a handler and write the session, or put the tag in the query.
* **No `Option<Locale>` extractor**, and no negotiation inside the extractor. A
route without the layer answers `500`.
* **No prelude entry.** Import from `arcature::i18n::`.
* **No `Deserialize` for `LocaleId`.** Take a `String` and `parse` it.
* **No `Content-Language` from a view unless you call `.in_locale(..)`**, and
no inference of a rendered page's language from the negotiated one.
* **No `{{ t("key") }}` template filter**, by the same argument that made the
view layer compile its templates.
* **No `locale` field in generated TypeScript.** The Inertia prop is added
after prop resolution and is not part of any page contract.
* **No number or date formatting API of its own.** `NUMBER` and `DATETIME`
inside a Fluent message are the whole surface; there is no
`locale.format_currency(..)`.
* **No diagnostics at all without `observe`.** Every failure becomes the same
generic `500`, and the reason is dropped rather than logged.
* **`LocaleNegotiator::resolve` truncates `accept_language` at 512 bytes on a
byte boundary.** Through `LocaleLayer` the value has already passed
`HeaderValue::to_str`, which admits only visible ASCII, so the cut is always
a character boundary. Calling `resolve` directly with a longer non-ASCII
string can land the cut mid-character and panic. The layer cannot reach it;
a test harness calling `resolve` by hand can.