Skip to main content

Module normalizer

Module normalizer 

Source
Expand description

Single-pass SIMD-guided normalizer implementations (NFC, NFD, NFKC, NFKD).

The core loop scans 64-byte chunks via SIMD to identify passthrough regions (all bytes below a form-dependent bound), copying them directly. Non-passthrough bytes trigger scalar decode + decompose + CCC sort + optional recomposition.

Structs§

NfcNormalizer
NFC normalizer: Canonical Decomposition, followed by Canonical Composition.
NfdNormalizer
NFD normalizer: Canonical Decomposition.
NfkcNormalizer
NFKC normalizer: Compatibility Decomposition, followed by Canonical Composition.
NfkdNormalizer
NFKD normalizer: Compatibility Decomposition.

Enums§

Form
Unicode normalization form.