Skip to main content

normalize_text

Function normalize_text 

Source
pub fn normalize_text(text: &str) -> String
Expand description

Normalize text by lowercasing and removing excess whitespace.

This is useful for text deduplication where case differences shouldn’t affect similarity.