Expand description
Symbols that parser works on
Structs§
Constants§
- HIDDEN
- Shorthand for TOKEN_HIDDEN_CHANNEL
- TOKEN_
DEFAULT_ CHANNEL - Default channel lexer emits tokens to
- TOKEN_
EOF - Type of EOF token
- TOKEN_
EPSILON - Type of tokens that DFA can use to advance to next state without consuming actual input token. Should not be created by downstream implementations.
- TOKEN_
HIDDEN_ CHANNEL - Predefined additional channel for lexer to assign tokens to
- TOKEN_
INVALID_ TYPE - Type of tokens that parser considers invalid
- TOKEN_
MIN_ USER_ TOKEN_ TYPE - Min token type that can be assigned to tokens created by downstream implementations.
Traits§
- Token
- Implemented by tokens that are produced by a
TokenFactory
Type Aliases§
- Common
Token - Most versatile Token that uses Cow to save data Can be used seamlessly switch from owned to zero-copy parsing
- Owning
Token - Token that owns its data