Function rnix::tokenizer::tokenize

source · 
pub fn tokenize(input: &str) -> Vec<Token<'_>>
Expand description

A convenience function for tokenizing the given input