Function pupil::lexer::tokenize

pub fn tokenize<'a>(input: &'a str) -> TokenIterator<'a>

Create a new TokenIterator for a string.