Skip to main content

lex

Function lex 

Source
pub fn lex(input: &str) -> Vec<Token>
Expand description

Lex input into a flat, lossless token stream, consulting only the built-in signature DB for verbatim commands/environments. The entry used by the first parse pass; lex_with adds user-defined verbatim commands. Uses the Document flavor (ordinary starting catcodes).