Module lexer 

Structs

Lexer

LexerTerminal

Output terminal emitted by the lexer.

Functions

tokenize_all

Tokenizes the entire text and returns a deque of terminals.