Enum TokenType 

pub enum TokenType {
    Word,
    Number(NumberType),
    String,
    Char,
    Symbol,
    Operator,
}

Represents all possible token types that can be parsed by the tokenizer

Variants

Word

Any alphanumeric string

Number(NumberType)

A numeric token, where NumberType specifies the format

String

A sequence of characters surrounded by double quotes (“example”)

Char

A single character surrounded by single quotes (‘a’)

Symbol

A character recognized as a symbol

Operator

A character recognized as an operator

Trait Implementations

impl Clone for TokenType

fn clone(&self) -> TokenType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for TokenType

fn eq(&self, other: &TokenType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for TokenType

impl Eq for TokenType

impl StructuralPartialEq for TokenType