Function ress::tokenize

source ·
pub fn tokenize(text: &str) -> Result<Vec<Token<&str>>, Error>
Expand description

a convince function for collecting a scanner into a Vec<Token>