Expand description
Auto-generated module
🤖 Generated with SplitRS
Structs§
- Annotated
Token - A token annotated with its depth and category.
- Operator
Priority - A priority level for operator tokens.
- Reformat
Options - Options for token stream reformatting.
- Rich
Token - A classified token enriched with operator metadata.
- Token
Meta - Rich metadata attached to a token.
- Token
Ngram Iter - A window iterator that yields consecutive N-grams of tokens.
- Token
Stream - A peekable stream of tokens with span tracking.
Enums§
- Bracket
Kind - Matching bracket pairs.
- Operator
Arity - Classify a TokenKind as unary, binary, or non-operator.
- Token
Category - High-level category of a token used for error messages and syntax highlighting.
- Token
Pattern - A simple pattern for matching token sequences.