pub fn lex(src: &str) -> Result<Vec<Token>, Error>Expand description
Lex the entire source string into a vector of tokens.
§Errors
Returns Error::UnexpectedChar on any non-ASCII byte or any character
outside the grammar, or Error::UnexpectedEnd if a multi-byte token
(such as :=) is truncated.
§Examples
use lambda_ref_cat::lexer::lex;
let tokens = lex("ref (\\x. x)")?;
assert_eq!(tokens.len(), 7);