Enum Token

pub enum Token {
    Illegal,
    Eof,
    Ident(String),
    Int(i32),
    String(String),
    Assign,
    Plus,
    Minus,
    Bang,
    Asterisk,
    Slash,
    Lt,
    Gt,
    Eq,
    NotEq,
    Comma,
    Semicolon,
    Colon,
    LParen,
    RParen,
    LBrace,
    RBrace,
    LBracket,
    RBracket,
    Function,
    Let,
    True,
    False,
    If,
    Else,
    Return,
}

Defines the tokens in the token stream generated by the lexer.

Variants

Illegal

An illegal, unrecognized token.

Eof

Token marking the end of a file stream.

Ident(String)

Identifier, e.g., add, foobar, x, y, …

Int(i32)

Integer literal, e.g., [0-9]

String(String)

A string literal, e.g., "Hello, world!"

Assign

Assignment operator =

Plus

Summation arithmetic operator +

Minus

Subtraction arithmetic operator -

Bang

Negation logical operator !

Asterisk

Multiplication arithmetic operator *

Slash

Division arithmetic operator /

Lt

Less than logical operator <

Gt

Greater than logical operator >

Eq

Equality logical operator ==

NotEq

Inequality logical operator !=

Comma

Comma ,

Semicolon

Semicolon ;

Colon

Colon :

LParen

Left parenthesis (

RParen

Right parenthesis )

LBrace

Left curly brace {{

RBrace

Right curly bracket }}

LBracket

Left square bracket [

RBracket

Right square bracket ]

Function

fn keyword

Let

let keyword

True

true keyword

False

false keyword

If

if keyword

Else

else keyword

Return

return keyword

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for Token

impl StructuralPartialEq for Token