pub fn tokenize(source: &str) -> Result<Vec<Token>, LexError>
Tokenize a source string into a Vec<Token>, halting at the first lexical error. Callers that need streaming or custom recovery should construct a Lexer directly.
Vec<Token>
Lexer
Returns the first LexError encountered while scanning source.
LexError
source