tokenize

Function tokenize 

Source
pub fn tokenize(string: &str, string_cache: &mut StringCache) -> Vec<Token>
Expand description

Converts the provided string to a Phrase.