Enum TokenValue 

pub enum TokenValue {
    None,
    Ident(usize),
    Number(i64),
    Comment(String),
    Stat {
        comments: Vec<String>,
        value: Option<i64>,
    },
}

The payload carried by a lexical token.

Tokens may or may not carry extra data depending on their kind. For example, identifiers and numbers store auxiliary information such as a symbol-table index or a literal integer value.

This payload is paired with a TokenID inside a CalcToken.

Variants

• TokenValue::None: No extra data (typical for punctuation or operators).

• [TokenValue::Ident(usize)]: The symbol table index for an identifier. The usize refers to an entry managed by the symbol table (see your crate’s SymTab type).

• [TokenValue::Number(i64)]: A parsed integer literal.

Example

// Construct a token representing a number
let token = TokenValue::Number(42);

// Ensure that it is a number, and extract its value
let TokenValue::Number(n) = token else {
    panic!("Expected a numeric token");
};

println!("Numeric literal: {n}");
assert_eq!(n, 42);

Variants

None

No associated data (for symbols or keywords).

Ident(usize)

Identifier token with an index into the symbol table.

Number(i64)

Integer literal token.

Comment(String)

Comment.

Stat

Statement

Fields

comments: Vec<String>

value: Option<i64>

Trait Implementations

impl Clone for TokenValue

fn clone(&self) -> TokenValue

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenValue

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.