Skip to main content

tokenize_lite

Function tokenize_lite 

Source
pub fn tokenize_lite(source: &str, language: Language) -> Vec<Token>
Expand description

Tokenizes a source file, dropping trivia.