ftml::tokenizer
pub fn tokenize<'t>(log: &Logger, text: &'t str) -> Tokenization<'t>
Take an input string and produce a list of tokens for consumption by the parser.