Function rslint_parser::tokenize[][src]

pub fn tokenize(text: &str, file_id: usize) -> (Vec<Token>, Vec<ParserError>)
Expand description

Run the rslint_lexer lexer to turn source code into tokens and errors produced by the lexer