Enum TokenType 

pub enum TokenType {
    Symbol(String),
    Identifier(String),
    StringLiteral(String),
    NumberLiteral(String, Number),
    Keyword(String),
    Comment(String),
    Ignore,
    NewLine,
    Eof,
    Unknown,
}

Variants

Symbol(String)

a symbol from the symbols list

Identifier(String)

an identifier

StringLiteral(String)

a string litteral. value is the string value, without the delimiting quotes

NumberLiteral(String, Number)

a number literal, with its string representation in the code and its parsed value

Keyword(String)

a keyword from the keywords list

Comment(String)

a single or multi-line comment. The value contains the delimiting characters.

Ignore

space, tabulations, …

NewLine

a newline character

Eof

Unknown

only if Scanner::run returns an error

Implementations

impl TokenType

pub fn len(&self) -> usize

Trait Implementations

impl Debug for TokenType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for TokenType

fn eq(&self, other: &TokenType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for TokenType