oxitext-icu — ICU4X-backed Unicode services for OxiText

oxitext-icu is the Unicode/CLDR services layer of OxiText. It provides CLDR-compliant text boundary analysis (line, word, grapheme-cluster, sentence), locale-aware collation (Unicode Collation Algorithm), case mapping, normalization (NFC/NFD/NFKC/NFKD), script detection and itemization, and locale-aware number/list/plural/date-time formatting. The CLDR line-break and word-boundary results feed directly into the layout engine in oxitext-layout.

This crate wraps the ICU4X family of crates (icu_segmenter, icu_collator, icu_casemap, icu_normalizer, icu_properties, icu_locale_core, icu_decimal, icu_list, icu_plurals, icu_datetime, fixed_decimal). ICU4X is the Pure-Rust successor to the C/C++ ICU library, so this crate is 100% Pure Rust — no C ICU, no libicu. Unicode/CLDR data is compiled in via ICU4X's compiled_data feature, which bakes the tables into the binary at build time.

Binary-size note. Compiled CLDR data adds roughly 5–15 MB to the final binary depending on which modules are exercised (icu_segmenter ≈ 1–3 MB, icu_collator ≈ 2–5 MB, others smaller). For size-sensitive targets, use ICU4X's icu_provider_blob or icu_provider_fs to load data at runtime instead of baking it in.

Installation

[dependencies]
oxitext-icu = "0.1.0"

This crate has no default features; every type is available as soon as the dependency is added.

Quick Start

use oxitext_icu::{IcuSegmenter, SegmentKind};

let seg = IcuSegmenter::new();
let breaks = seg.break_points("Hello world", SegmentKind::Word);
assert!(breaks.len() >= 2);

Rich segmentation with byte offsets

use oxitext_icu::{IcuSegmenter, SegmentKind};

let seg = IcuSegmenter::new();
let segs = seg.segments("Hello world", SegmentKind::Word);
let words: Vec<&str> = segs.iter()
    .filter(|s| !s.text.trim().is_empty())
    .map(|s| s.text.as_str())
    .collect();
assert!(words.contains(&"Hello"));
assert!(words.contains(&"world"));

Locale-aware collation

use oxitext_icu::{IcuCollator, CollationStrength};
use std::cmp::Ordering;

// Primary strength: accents and case are ignored.
let c = IcuCollator::with_strength("en", CollationStrength::Primary)?;
assert_eq!(c.compare("Apple", "apple"), Ordering::Equal);

// Swedish: "z" sorts before "ä".
let sv = IcuCollator::new_for_locale("sv")?;
assert_eq!(sv.compare("z", "ä"), Ordering::Less);
# Ok::<(), oxitext_icu::CollateError>(())

API Overview

Segmentation — segment module

Collation — collate module

Case mapping — casemap module

Normalization — normalize module

Character properties — properties module

Formatters — number, list, plural, datetime modules

Error type

IcuError is a crate-root type alias for CollateError, shared across the formatter constructors.

Cross-references

• oxitext-layout — consumes the CLDR break offsets from cldr_line_breaks / line_break_opportunities for word wrapping.
• oxitext — re-exports a curated subset of this crate under oxitext::icu behind the icu feature.
• oxitext-core — shared text/style types.
• oxitext-shape · oxitext-raster · oxitext-sdf · oxitext-bench — sibling crates in the OxiText pipeline.

License

Apache-2.0 — COOLJAPAN OU (Team Kitasan)