Skip to main content

normalize

Function normalize 

Source
pub fn normalize(text: &str) -> String
Expand description

Canonicalize text’s Khmer encoding before segmentation. Idempotent: normalizing already-canonical (or already-normalized) text is a no-op. Byte-length-preserving: only reorders characters, never adds or removes any, so span offsets over the original text stay valid.