Enum TokenType

pub enum TokenType {
    Keyword(Keyword),
    Ident {
        start: usize,
        end: usize,
    },
    Num(i32),
    Str {
        start: usize,
        end: usize,
    },
    Bool(bool),
    Comment {
        start: usize,
        end: usize,
    },
    Discard,
    Punct(Punct),
    Term,
}

The various types of tokens.

Comments, strings and identifiers keep the range of their values within the source code. The start of that range is the index of the first character of the token. The end is the index after the last character of the token.

Variants

Keyword(Keyword)

Ident

Fields

start: usize

end: usize

Num(i32)

Str

Fields

start: usize

end: usize

Bool(bool)

Comment

Fields

start: usize

end: usize

Discard

Punct(Punct)

Term

Implementations

impl TokenType

pub fn type_eq(self, other: TokenType) -> bool

Trait Implementations

impl Clone for TokenType

fn clone(&self) -> TokenType

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for TokenType

fn eq(&self, other: &TokenType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for TokenType

impl StructuralPartialEq for TokenType