Skip to main content

Module lexer

Module lexer 

Source

Functions§

classify_keyword
Classify an identifier string as a keyword or plain IDENT.
lex
Lex source into a sequence of (SyntaxKind, slice) pairs.
lex_punctuation
Try to lex punctuation starting at pos. Returns (kind, advance). Handles all greedy multi-char disambiguation. None means the byte isn’t punctuation at all (falls through to ident/digit/error handling).