Function ress::tokenize

pub fn tokenize(text: &str) -> Result<Vec<Token<&str>>, Error>

a convince function for collecting a scanner into a Vec<Token>