Enum TokenType 

pub enum TokenType {
    EOF,
    SPACE,
    TAB,
    NEWLINE,
    CR,
    FORM_FEED,
    NOT,
    AND,
    OR,
    BETWEEN,
    LIKE,
    ESCAPE,
    IN,
    IS,
    TRUE,
    FALSE,
    NULL,
    EQ,
    NE,
    GT,
    GE,
    LT,
    LE,
    LPAREN,
    COMMA,
    RPAREN,
    PLUS,
    MINUS,
    STAR,
    SLASH,
    PERCENT,
    LINE_COMMENT,
    BLOCK_COMMENT,
    DECIMAL_LITERAL,
    HEX_LITERAL,
    OCTAL_LITERAL,
    FLOATING_POINT_LITERAL,
    STRING_LITERAL,
    ID,
    INVALID,
}

Token type enumeration

Represents all possible token types recognized by the lexer.

Variants

EOF

End of file

SPACE

Whitespace: space

TAB

Whitespace: tab

NEWLINE

Whitespace: newline

CR

Whitespace: carriage return

FORM_FEED

Whitespace: form feed

NOT

Keyword: NOT

AND

Keyword: AND

OR

Keyword: OR

BETWEEN

Keyword: BETWEEN

LIKE

Keyword: LIKE

ESCAPE

Keyword: ESCAPE

IN

Keyword: IN

IS

Keyword: IS

TRUE

Keyword: TRUE

FALSE

Keyword: FALSE

NULL

Keyword: NULL

EQ

Operator: = (equals)

NE

Operator: <> or != (not equals)

GT

Operator: > (greater than)

GE

Operator: >= (greater than or equal)

LT

Operator: < (less than)

LE

Operator: <= (less than or equal)

LPAREN

Delimiter: ( (left parenthesis)

COMMA

Delimiter: , (comma)

RPAREN

Delimiter: ) (right parenthesis)

PLUS

Operator: + (plus)

MINUS

Operator: - (minus)

STAR

Operator: * (multiply)

SLASH

Operator: / (divide)

PERCENT

Operator: % (modulo)

LINE_COMMENT

Line comment

BLOCK_COMMENT

Block comment

DECIMAL_LITERAL

Decimal numeric literal

HEX_LITERAL

Hexadecimal numeric literal

OCTAL_LITERAL

Octal numeric literal

FLOATING_POINT_LITERAL

Floating point numeric literal

STRING_LITERAL

String literal (single-quoted)

ID

Identifier

INVALID

Invalid token marker

Trait Implementations

impl Clone for TokenType

fn clone(&self) -> TokenType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for TokenType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for TokenType

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for TokenType

fn eq(&self, other: &TokenType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for TokenType

impl Eq for TokenType

impl StructuralPartialEq for TokenType