Skip to main content

normalize_tail

Function normalize_tail 

Source
pub fn normalize_tail(decomposed: &str) -> String
Expand description

Port of the tail of Words2Num_Base._normalize — the pure ASCII-shaped rewriting that follows NFKD decomposition + combining-mark stripping.

text = text.lower().replace("_", " ")
text = re.sub(r"(?<=[a-z])-(?=[a-z])", " ", text)
text = re.sub(r"[,;:!\?\"']", " ", text)
text = re.sub(r"\.(?!\d)", " ", text)
text = re.sub(r"\s+", " ", text).strip()