Enum TokenKind 

pub enum TokenKind {
    Fn,
    Pub,
    Let,
    Mut,
    Struct,
    Mod,
    If,
    Else,
    While,
    Break,
    Continue,
    Return,
    Enum,
    Impl,
    Match,
    For,
    In,
    SelfKw,
    CrateKw,
    SuperKw,
    UseKw,
    As,
    Async,
    Await,
    ReservedKeyword(String),
    Identifier(String),
    IntegerLiteral(String),
    FloatLiteral(String),
    StringLiteral(String),
    BoolLiteral(bool),
    Lifetime(String),
    Plus,
    Minus,
    Star,
    Slash,
    Percent,
    Eq,
    EqEq,
    NotEq,
    Lt,
    LtEq,
    Gt,
    GtEq,
    Bang,
    AmpAmp,
    PipePipe,
    Pipe,
    Amp,
    ColonColon,
    Colon,
    Semi,
    Comma,
    Dot,
    DotDot,
    DotDotEq,
    LParen,
    RParen,
    LBrace,
    RBrace,
    LBracket,
    RBracket,
    Arrow,
    FatArrow,
    Question,
    Hash,
}

The kind of a lexical token. Keywords that are part of the Varyk surface (spec 4.1) each get their own variant; every other Rust keyword and reserved word, including the 2024-edition ones, lexes as TokenKind::ReservedKeyword carrying its text, so the parser can report it by name.

Variants

Fn

Pub

Let

Mut

Struct

Mod

If

Else

While

Break

Continue

Return

Enum

Impl

Match

For

In

SelfKw

self, spec 2.1, also a path prefix (spec 3.1, 3.3). Self is unrelated and still lexes as TokenKind::ReservedKeyword: Varyk has no way to write it.

CrateKw

crate, a path prefix naming the crate root (spec 3.1, 3.3).

SuperKw

super, a path prefix naming the parent module (spec 3.1, 3.3).

UseKw

use, spec 3.3.

As

as: a cast, expr as T (M4 spec 2.9), and the alias in use path as name; (spec 3.3).

Async

async, before fn (milestone 5b1 spec 2.2).

Await

await, only after a . (milestone 5b1 spec 2.3).

ReservedKeyword(String)

Any other Rust keyword or reserved word, carrying its text.

Identifier(String)

IntegerLiteral(String)

Raw source text of an integer literal, digits only.

FloatLiteral(String)

Raw source text of a float literal, digits . digits.

StringLiteral(String)

Raw source text between the quotes of a string literal, unmodified.

BoolLiteral(bool)

Lifetime(String)

A 'ident lifetime, carrying the identifier without the quote.

Plus

Minus

Star

Slash

Percent

Eq

EqEq

NotEq

Lt

LtEq

Gt

GtEq

Bang

AmpAmp

PipePipe

Pipe

A lone |, around a closure’s parameters (M4 spec 2.2).

Amp

ColonColon

Colon

Semi

Comma

Dot

DotDot

DotDotEq

..=, an inclusive range (M4 spec 2.5, 2.11).

LParen

RParen

LBrace

RBrace

LBracket

RBracket

Arrow

FatArrow

Question

Hash

#, which starts an attribute, #[name] or #[name(literal)] (M5a spec 2.1, 2.2).

Trait Implementations

impl Clone for TokenKind

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenKind

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for TokenKind

impl PartialEq for TokenKind

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for TokenKind