Enum Token

pub enum Token {
    Identifier(String),
    Assign,
    Plus,
    Minus,
    Number(f64),
    EOF,
}

Token module for lexical analysis.

This module defines the tokens used in the R2Rust interpreter and provides functionality for tokenizing input strings. Represents the different types of tokens in the interpreter.

Variants

Identifier(String)

An identifier, such as variable names.

Assign

Assignment operator (<-).

Plus

Addition operator (+).

Minus

Subtraction operator (-).

Number(f64)

A numeric literal (e.g., integers or floating-point numbers).

EOF

End of file token.

Trait Implementations

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Token