Expand description
Used for tokenizing, the process of converting the code to individual tokens. Useful for getting symbols and manually tokenizing without going using an AST.
Structs
- Used to represent exact positions of tokens in code
- A reference to a token used by Ast’s. Dereferences to a
Token
- Information about an error that occurs while tokenizing
Enums
- InterpolatedStringKind
roblox
Whether or not this section is the beginning, middle, end, or if this is a standalone string. - The types of quotes used in a Lua string
- A literal symbol, used for both words important to syntax (like while) and operators (like +)
- The kind of token. Contains no additional data.
- The type of tokens in parsed code
- The possible errors that can happen while tokenizing.
Functions
- Returns a list of tokens. You probably want
parse
instead.