Skip to main content

tokenize_query

Function tokenize_query 

Source
pub fn tokenize_query(raw: &str) -> Vec<String>
Expand description

Tokenize raw user text: split on whitespace/punctuation, lowercase, keep alnum/_/-/:/.