Skip to main content

text_collapse

Function text_collapse 

Source
pub fn text_collapse(text: &str) -> String
Expand description

Normalize and simplify text for similarity hashing.

Applies NFD normalization, lowercasing, removes whitespace and characters in Unicode categories C (control), M (mark), and P (punctuation), then recombines with NFKC normalization.