Skip to main content

significant_tokens

Function significant_tokens 

Source
pub fn significant_tokens(tokens: &[String]) -> Vec<String>
Expand description

Strip stopwords; if nothing remains, fall back to tokens longer than 1 char.