Enum pattern_lexer::token::TokenKind

pub enum TokenKind<'a> {
    KEYWORD(&'a str),
    DELIMITER(&'a str),
    PUNCTUATION(&'a str),
    OPERATOR(&'a str),
    COMMENT(&'a str),
    WHITESPACE(&'a str),
    LITERAL(&'a str),
    IDENTIFIER,
}

Premade Token kinds for semantic (examples are not mandatory): | rust TokenKind | Explanation | Examples | |————————|————————————|–––––––––––| | rust KEYWORD | Reserved words | if, return, … | | rust DELIMITER | Paired delimiter symbols | (), [], {}, … | | rust PUNCTUATION | Punctuation symbols | ;, ., … | | rust OPERATOR | Symbols that operates on arguments | +, -, =, … | | rust COMMENT | Line or block comments | //, /* … */, … | | rust WHITESPACE | Non-printable characters | - | | rust LITERAL | Numerical, logical, textual values | 1, true, “true”, … | | rust IDENTIFIER | Names assigned in a program | x, temp, PRINT | These token kinds (except rust IDENTIFIER) should be constructed with a name that can be used to differentiate tokens with same kind.

Variants

KEYWORD(&'a str)

Ordered from high priority to low priority.

DELIMITER(&'a str)

PUNCTUATION(&'a str)

OPERATOR(&'a str)

COMMENT(&'a str)

WHITESPACE(&'a str)

LITERAL(&'a str)

IDENTIFIER

Trait Implementations

impl<'a> Clone for TokenKind<'a>

fn clone(&self) -> TokenKind<'a>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a> Debug for TokenKind<'a>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'a> PartialEq for TokenKind<'a>

fn eq(&self, other: &TokenKind<'a>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<'a> PartialOrd for TokenKind<'a>

fn partial_cmp(&self, other: &TokenKind<'a>) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

This method tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

This method tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

This method tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

This method tests greater than or equal to (for self and other) and is used by the >= operator.

impl<'a> StructuralPartialEq for TokenKind<'a>