Skip to main content

tokens

Function tokens 

Source
pub fn tokens(text: &str) -> BTreeSet<String>
Expand description

Significant words, lowercased. Punctuation goes, short words go, and words that appear in every bug report go.