Trait implemented for an enum representing all tokens. You should never have
to implement it manually, use the
#[derive(Logos)] attribute on your enum.
Extras for the particular lexer. This can be set using
#[logos(extras = MyExtras)] and accessed inside callbacks.
Source type this token can be lexed from. This will default to
unless one of the defined patterns explicitly uses non-unicode byte values
or byte slices, in which case that implementation will use
SIZE is simply a number of possible variants of the
Logos enum. The
derive macro will make sure that all variants don't hold values larger
or equal to
This can be extremely useful for creating
Logos Lookup Tables.
const ERROR: Self
const of the variant marked as
The heart of Logos. Called by the
Lexer. The implementation for this function
is generated by the
Create a new instance of a
Lexer that will produce tokens implementing