Expand description
Validates the tokens generated by the tokenizer.
Structs§
- Token
Iterator - Yields tokens, while maintaining the byte offset of the latest token.
Enums§
- Validation
Error - Error class raised by the validator.
Validates the tokens generated by the tokenizer.