Enum Token 

pub enum Token {
    And,
    Break,
    Do,
    Else,
    Elseif,
    End,
    False,
    For,
    Function,
    Global,
    Goto,
    If,
    In,
    Local,
    Nil,
    Not,
    Or,
    Repeat,
    Return,
    Then,
    True,
    Until,
    While,
    Plus,
    Minus,
    Star,
    Slash,
    DSlash,
    Percent,
    Caret,
    Hash,
    Amp,
    Tilde,
    Pipe,
    Shl,
    Shr,
    Eq,
    Ne,
    Le,
    Ge,
    Lt,
    Gt,
    Assign,
    LParen,
    RParen,
    LBrace,
    RBrace,
    LBracket,
    RBracket,
    DColon,
    Semi,
    Colon,
    Comma,
    Dot,
    Concat,
    Ellipsis,
    Int(i64),
    Float(f64),
    Str(Vec<u8>),
    Name(Box<str>),
    At,
    MacroBraceOpen,
    MacroBraceClose,
    MacroQuote(Box<[TokenInfo]>),
    Eof,
}

One lexical token produced by the lexer.

Variants

And

and keyword.

Break

break keyword.

Do

do keyword.

Else

else keyword.

Elseif

elseif keyword.

End

end keyword.

False

false keyword.

For

for keyword.

Function

function keyword.

Global

5.5 global keyword.

Goto

goto keyword.

If

if keyword.

In

in keyword.

Local

local keyword.

Nil

nil keyword.

Not

not keyword.

Or

or keyword.

Repeat

repeat keyword.

Return

return keyword.

Then

then keyword.

True

true keyword.

Until

until keyword.

While

while keyword.

Plus

+ symbol.

Minus

- symbol.

Star

* symbol.

Slash

/ symbol.

DSlash

// symbol (floor division).

Percent

% symbol.

Caret

^ symbol.

Hash

# symbol.

Amp

& symbol.

Tilde

~ symbol (bitwise xor / unary bnot).

Pipe

| symbol.

Shl

<< symbol.

Shr

>> symbol.

Eq

== symbol.

Ne

~= symbol.

Le

<= symbol.

Ge

>= symbol.

Lt

< symbol.

Gt

> symbol.

Assign

= symbol (assignment).

LParen

( symbol.

RParen

) symbol.

LBrace

{ symbol.

RBrace

} symbol.

LBracket

[ symbol.

RBracket

] symbol.

DColon

:: symbol (label delimiter).

Semi

; symbol.

Colon

: symbol.

Comma

, symbol.

Dot

. symbol.

Concat

.. symbol (concatenation).

Ellipsis

... symbol (vararg).

Int(i64)

Integer literal.

Tuple Fields

0: i64

Decoded 64-bit signed value.

Float(f64)

Floating-point literal.

Tuple Fields

0: f64

Decoded IEEE-754 double value.

Str(Vec<u8>)

String literal (raw bytes; Lua strings are 8-bit clean).

Tuple Fields

0: Vec<u8>

Decoded byte contents.

Name(Box<str>)

Identifier.

Tuple Fields

0: Box<str>

Source text of the identifier.

At

MacroLua @ sigil (v1.3 Phase ML). Lexed only when version.is_macro_lua(); PUC 5.1-5.5 sources continue to error unexpected symbol near '@'.

MacroBraceOpen

MacroLua explicit quote-block opener @{. Lexed only when version.is_macro_lua(); pairs with Token::MacroBraceClose.

MacroBraceClose

MacroLua explicit quote-block closer }@. Lexed only when version.is_macro_lua(); pairs with Token::MacroBraceOpen.

MacroQuote(Box<[TokenInfo]>)

Synthetic token produced by the macro expander pre-pass: a captured token run (the body of a @quote{...} or @{...}@ block). The lexer never emits this. After the expander runs it splices these back into the stream as raw token sequences before the parser proper sees them.

Tuple Fields

0: Box<[TokenInfo]>

Captured token run.

Eof

End-of-file marker.

Implementations

impl Token

pub fn describe(&self, src: &[u8], span: Span, version: LuaVersion) -> String

The near-token shown in ... near <tok> error messages. Every concrete token (names, literals, symbols, keywords) is wrapped in single quotes per PUC’s txtToken/luaX_token2str. The pseudo-token <eof> is unquoted under 5.2+ — those suites’ checksyntax has a ^<%a guard that adds quotes only when the expected token doesn’t already start with <. 5.1’s checksyntax has no such guard and unconditionally wraps the expected token, so <eof> must come through quoted there to match the ... near '<eof>' shape (5.1 errors.lua :20-:21 pin this; PUC’s 5.1 luaX_lexerror output is the same).

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Token