pub fn tokenize_complete(input: &str) -> Box<[Token<'_>]>
Returns an owned array containing all of the tokens produced by the tokenizer. This allows us to backtrack in case of an error.