Function ucglib::tokenizer::tokenize
[−]
[src]
pub fn tokenize(input: Span) -> Result<Vec<Token>, (Position, ErrorKind)>
Consumes an input Span and returns either a Vec