Function rslint_parser::tokenize

pub fn tokenize(text: &str, file_id: usize) -> (Vec<Token>, Vec<ParserError>)

Run the rslint_lexer lexer to turn source code into tokens and errors produced by the lexer