Enum TokenKind 

pub enum TokenKind {
    NumberLiteral(f64),
    BigIntLiteral(String),
    StringLiteral(String),
    BooleanLiteral(bool),
    NullLiteral,
    TemplateNoSubstitution(String),
    TemplateHead(String),
    TemplateMiddle(String),
    TemplateTail(String),
    RegexLiteral {
        source: String,
        flags: String,
    },
    Identifier,
    Let,
    Const,
    Function,
    If,
    Else,
    While,
    Do,
    For,
    Break,
    Continue,
    Return,
    Switch,
    Case,
    Default,
    Export,
    Void,
    Interface,
    Enum,
    In,
    Typeof,
    Import,
    Instanceof,
    New,
    Try,
    Catch,
    Finally,
    Throw,
    Class,
    Extends,
    Implements,
    Super,
    This,
    Plus,
    Minus,
    Star,
    Slash,
    Percent,
    Equals,
    PlusEquals,
    MinusEquals,
    StarEquals,
    SlashEquals,
    PercentEquals,
    StarStar,
    StarStarEquals,
    PlusPlus,
    MinusMinus,
    EqEqEq,
    EqEq,
    BangEqEq,
    BangEq,
    LessThan,
    GreaterThan,
    LessEquals,
    GreaterEquals,
    Bang,
    AmpAmp,
    Amp,
    AmpEquals,
    PipeEquals,
    Caret,
    CaretEquals,
    Tilde,
    Pipe,
    PipePipe,
    Dot,
    DotDotDot,
    Arrow,
    Question,
    QuestionDot,
    QuestionQuestion,
    LeftParen,
    RightParen,
    LeftBrace,
    RightBrace,
    LeftBracket,
    RightBracket,
    Comma,
    Colon,
    Semicolon,
    Newline,
    Eof,
}

Variants

NumberLiteral(f64)

BigIntLiteral(String)

No n suffix, no leading sign (unary - is at the AST level). String because the value may exceed any fixed-width integer.

StringLiteral(String)

BooleanLiteral(bool)

NullLiteral

TemplateNoSubstitution(String)

No ${…} interpolations; parser collapses to a plain string literal.

TemplateHead(String)

Cooked text up to the first ${. Lexer pushes a frame onto template_frames; the next depth-0 } re-enters template scanning.

TemplateMiddle(String)

TemplateTail(String)

Cooked text from the last } to the closing backtick. Pops the frame pushed by TemplateHead.

RegexLiteral

source preserves escape sequences verbatim (the regex engine interprets them, not the lexer). Context disambiguation (literal vs. division) via is_regex_context.

Fields

source: String

flags: String

Identifier

Let

Const

Function

If

Else

While

Do

For

Break

Continue

Return

Switch

Case

Default

Export

Phase-1 forgiveness: accepted as a leading modifier on top-level declarations and ignored (there is no export consumer yet).

Void

Interface

Enum

In

Also drives the "field" in x narrowing predicate.

Typeof

Import

Instanceof

New

Try

Catch

Finally

Throw

Class

Extends

Implements

Super

This

Plus

Minus

Star

Slash

Percent

Equals

PlusEquals

MinusEquals

StarEquals

SlashEquals

PercentEquals

Lowered by the compound-assignment desugar to x = x % y.

StarStar

Right-associative, higher precedence than *///%. Lex order: **= matched before ** before *=.

StarStarEquals

PlusPlus

Postfix-only (no prefix ++x). Lex order: ++ matched before +=.

MinusMinus

EqEqEq

EqEq

BangEqEq

BangEq

LessThan

GreaterThan

LessEquals

GreaterEquals

Bang

AmpAmp

Amp

AmpEquals

PipeEquals

Caret

CaretEquals

Tilde

Pipe

PipePipe

Dot

DotDotDot

Parser accepts only inside destructuring patterns ({ … } / [ … ]).

Arrow

Question

QuestionDot

Single token so the postfix-chain parser dispatches on one peek.

QuestionQuestion

LeftParen

RightParen

LeftBrace

RightBrace

LeftBracket

RightBracket

Comma

Colon

Semicolon

Newline

Eof

Trait Implementations

impl Clone for TokenKind

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenKind

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for TokenKind

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for TokenKind