Enum NormalizedToken 

pub enum NormalizedToken {
    Keyword(String),
    Variable(usize),
    IntLiteral,
    FloatLiteral,
    StrLiteral,
    CharLiteral,
    BoolLiteral(bool),
    TypePlaceholder,
    Operator(String),
    Delimiter(char),
}

Normalized token representation for deduplication

Variants

Keyword(String)

Rust keyword (fn, let, if, match, etc.)

Variable(usize)

Placeholder for user-defined variable (e.g., var_1)

IntLiteral

Placeholder for literal values

FloatLiteral

StrLiteral

CharLiteral

BoolLiteral(bool)

TypePlaceholder

Type placeholder

Operator(String)

Operator or punctuation

Delimiter(char)

Delimiter (parentheses, braces, brackets)

Trait Implementations

impl Clone for NormalizedToken

fn clone(&self) -> NormalizedToken

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for NormalizedToken

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for NormalizedToken

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl Ord for NormalizedToken

fn cmp(&self, other: &NormalizedToken) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

where Self: Sized,

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

where Self: Sized,

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

where Self: Sized,

Restrict a value to a certain interval.

impl PartialEq for NormalizedToken

fn eq(&self, other: &NormalizedToken) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PartialOrd for NormalizedToken

fn partial_cmp(&self, other: &NormalizedToken) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

Tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

Tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

Tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

Tests greater than or equal to (for self and other) and is used by the >= operator.

impl Eq for NormalizedToken

impl StructuralPartialEq for NormalizedToken