Functions§
- classify_
keyword - Classify an identifier string as a keyword or plain
IDENT. - lex
- Lex
sourceinto a sequence of(SyntaxKind, slice)pairs. - lex_
punctuation - Try to lex punctuation starting at
pos. Returns(kind, advance). Handles all greedy multi-char disambiguation.Nonemeans the byte isn’t punctuation at all (falls through to ident/digit/error handling).