Skip to main content

lex

Function lex 

Source
pub fn lex(source: &str) -> Vec<LexToken>
Expand description

Lexes source text into tokens while preserving trivia.

Args: source: Raw Onlyfile source text.

Returns: Token stream including whitespace, comments and EOF.