Expand description
A lexer that reads Lua code and produces tokens. The crate provide two different lexers:
FastLexer: skips all the whitespace tokensFullLexer: produces every tokens
Structs§
- Fast
Tokenizer - Full
Tokenizer - Lexer
- This struct wraps the tokenizer to offer a simple interface to parse a string.
- Token
- A small struct that contains minimal information about a slice of the input. The token does not own the part of the code that it represents, it only keeps a reference to the original input.
Enums§
- Lexer
Error Type - Errors that can happen while tokenizing some input.
- Token
Type - The different type of tokens that will be produced by the lexer
Type Aliases§
- Fast
Lexer - A lexer that skips all the information about whitespaces.
- Full
Lexer - A lexer that keeps the information about whitespaces.
- Lexer
Error - When the lexer encounters an error, it returns the error type and the rest of the given input that was not parsed.