Enum TokenKind 

pub enum TokenKind {
    Eof,
    Newline,
    Indent,
    Dedent,
    Ident,
    String(Vec<(u32, u32)>),
    SingleString,
    MlString(Vec<(u32, u32)>),
    MlSingleString,
    Integer,
    Float,
    True,
    False,
    Null,
    Let,
    Env,
    Secret,
    Assign,
    Dot,
    Comma,
    QuestionColon,
    Plus,
    Minus,
    Star,
    Slash,
    EqEq,
    NotEq,
    Lt,
    LtEq,
    Gt,
    GtEq,
    AndAnd,
    OrOr,
    Bang,
    LBracket,
    RBracket,
    LBrace,
    RBrace,
    LParen,
    RParen,
    Colon,
    At,
    Unit(String),
}

Variants

Eof

Newline

Indent

Opened an indented block (spec/dual-syntax.md §4).

Dedent

Closed an indented block.

Ident

String(Vec<(u32, u32)>)

Double-quoted string: escapes processed, ${…} interpolated. Carries the source position (line offset, column) of every decoded character, exactly like MlString: nested interpolation inside sub-lexed expression text needs the same position fidelity.

SingleString

Single-quoted string: escapes processed, ${…} literal (spec §2.5).

MlString(Vec<(u32, u32)>)

Triple-double-quoted multi-line string: escapes processed, ${…} interpolated, common-indent stripping at parse time (spec §1.4.1). Carries the source position (line offset, column) of every decoded character: escapes decode to characters of different source width (\n is two source columns becoming one decoded newline on the same physical line), so only a per-character map keeps diagnostics on their true positions.

MlSingleString

Triple-single-quoted multi-line string: fully literal — backslashes kept verbatim, ${…} never interpolated (spec §1.4.1).

Integer

Float

True

False

Null

Let

Env

Secret

Assign

Dot

Comma

QuestionColon

Plus

Minus

Star

Slash

EqEq

NotEq

Lt

LtEq

Gt

GtEq

AndAnd

OrOr

Bang

LBracket

RBracket

LBrace

RBrace

LParen

RParen

Colon

At

Unit(String)

Trait Implementations

impl Clone for TokenKind

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenKind

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for TokenKind

impl PartialEq for TokenKind

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for TokenKind