db-collation
Offline, exact comparison of strings using an explicitly identified database collation and version.
db-collation reproduces the exact ordering semantics of a source database's
collation locally. It is a pure function of a collation description and the
input bytes. It opens no sockets, holds no database drivers, and never issues a
query.
Why this exists
Many applications need to order strings the same way a database does, but locally — for sorting cached values, comparing keys, or checking ranges — without a round trip to that database. The obvious approaches are not safe. A column may use a collated text type whose ordering differs from Rust's default bytewise comparison and from a generic Unicode collator, so results computed locally can silently disagree with the source.
Approximations are not enough:
| Same locale, different ICU | Result |
|---|---|
| PostgreSQL ICU 76 vs local ICU 78 | 4,704 mismatches / 64,516 ordered pairs, in each of 8 configurations |
MySQL utf8mb4_0900_ai_ci vs generic ICU |
928 mismatches / 8,257 sampled pairs |
MySQL utf8mb4_unicode_ci vs generic ICU |
1,664 mismatches / 8,257 sampled pairs |
Even the same locale name with a different ICU data version is unsafe: PostgreSQL
placed あ before U+11DB0, while a newer ICU reversed it. Selecting the same
locale name does not establish compatibility.
Design: exact or refused
This crate has one rule: reproduce a configuration exactly, or refuse it.
- For a supported configuration,
Collation::comparematches the source's comparison operator exactly (validated by the differential harness inharness/). - Anything not explicitly modelled returns
Error::Unsupported(orError::VersionMismatch/Error::FeatureDisabled). The caller falls back to comparing on the source database. - Version is part of the identity. PostgreSQL's ICU collation is gated on the
collversionreported by the catalog; MySQL collations are keyed by pinned UCA weight tables.
Supported configurations
| Engine | Configuration | Notes |
|---|---|---|
| Any | binary / C / POSIX |
bytewise; always available |
| PostgreSQL | ICU provider, UTF-8, deterministic | ucol_strcollUTF8 + bytewise tie-break |
| PostgreSQL | ICU provider, UTF-8, nondeterministic | no tie-break |
| MySQL | utf8mb4_0900_ai_ci |
UCA 9.0.0, NO PAD |
| MySQL | utf8mb4_unicode_ci |
UCA 4.0.0, PAD SPACE |
| MySQL | utf8mb4_0900_bin |
NO PAD, code-point order (equals bytewise for UTF-8) |
Everything else is refused: PostgreSQL libc/builtin providers and custom
collicurules, MySQL language-tailored *_0900_* collations,
utf8mb4_general_ci, utf8mb4_bin (PAD SPACE), other UCA versions, and
non-UTF-8 encodings.
Installation
[]
= "0.1"
Default features enable the MySQL UCA backend. The PostgreSQL ICU backend links ICU4C and is opt-in:
[]
= { = "0.1", = ["postgres-icu", "serde"] }
Usage
use ;
use Ordering;
compare and equal return Result: a backend failure is surfaced, never
silently reported as equality.
PostgreSQL ICU, with the source version supplied by the caller:
use ;
let c = postgres_icu?;
let _ = c.compare?;
# Ok::
The caller is responsible for reading the source catalog (provider, locale,
determinism, collversion) and passing it in. See
docs/CONFIGURATION.md.
Thread safety
Collation is Send + Sync. ICU4C collator handles are neither, so they are
cached per thread; the handle is never shared across threads.
Testing
Correctness is established by a differential harness that compares this crate (not a reimplementation) against live PostgreSQL and MySQL:
- 0 mismatches across PostgreSQL 14–18 (ICU 72/76) and MySQL 8.0/8.4/9.4, over a 1,414-string scenario-class corpus.
- Exact over the full BMP: the candidate's total order matches live MySQL for
every one of the 63,498 BMP scalar values, for each modelled collation
(
make harness-bmp). - The PostgreSQL candidate is built and run inside a candidate image derived from the oracle image, so it links the identical ICU data version.
- Unsupported configurations are asserted to be refused, and negative controls prove the differ can fail. A runtime comparison failure is never accepted as a refusal.
See harness/README.md.
Documentation
docs/ARCHITECTURE.md— how the backends work.docs/CONFIGURATION.md— supported matrix and version gates.docs/DESIGN_DECISIONS.md— why the API looks like this.docs/DEVELOPMENT.md— building, testing, tooling.SECURITY.md— reporting vulnerabilities and correctness bugs.CHANGELOG.md
Minimum Supported Rust Version
Rust 1.85 (edition 2024). MSRV increases are treated as minor-version bumps.
License
Licensed under the Apache License, Version 2.0. The MySQL weight tables are
generated from the Unicode UCA data (Unicode License v3). See
harness/data/PROVENANCE.md and
THIRD-PARTY-NOTICES.
Acknowledgments
Thanks to the projects and data this crate builds on:
- TiKV — while the weight tables are now
generated directly from the Unicode UCA data, TiKV's
utf8mb4collation implementation was an invaluable reference for reproducing MySQL's semantics (implicit-weight ranges, expansion handling, and PAD SPACE behaviour). Thank you to the TiKV authors. - Unicode Consortium — for the Collation
Algorithm and the
allkeysdata files. - ICU — the PostgreSQL backend links ICU4C via
rust_icu.
Contributing
See CONTRIBUTING.md. Unless you explicitly state otherwise,
any contribution intentionally submitted for inclusion in this crate is
licensed as above, without additional terms.