Enum hamster_wheel::tokenizer::tokenizer::Token

pub enum Token {
    Loop,
    Output,
    Import,
    End,
    As,
    LeftParentheses,
    RightParentheses,
    LeftBracket,
    RightBracket,
    LeftMustache,
    RightMustache,
    DoubleDot,
    Dot,
    Comma,
    Colon,
    SemiColon,
    Number,
    Variable,
    EOF,
    Error,
}

Variants

LoopOutputImportEndAsLeftParenthesesRightParenthesesLeftBracketRightBracketLeftMustacheRightMustacheDoubleDotDotCommaColonSemiColonNumberVariableEOFError

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl PartialEq<Token> for Token

fn eq(&self, other: &Token) -> bool

This method tests for self and other values to be equal, and is used by ==.

#[must_use] fn ne(&self, other: &Rhs) -> bool

This method tests for !=.

impl Debug for Token

fn fmt(&self, f: &mut Formatter) -> Result

Formats the value using the given formatter.

impl<'source, Source: Source<'source>> WithSource<Source> for Token

impl Logos for Token

type Extras = ()

Associated type Extras for the particular lexer. Those can handle things that aren't necessarily tokens, such as comments or Automatic Semicolon Insertion in JavaScript.

const SIZE: usize

SIZE is simply a number of possible variants of the Logos enum. The derive macro will make sure that all variants don't hold values larger or equal to SIZE.

const ERROR: Self

Helper const of the variant marked as #[error].

const END: Self

Helper const of the variant marked as #[end].

fn lexicon<'lexicon, 'source, Source>(
) -> &'lexicon Lexicon<Lexer<Self, Source>> where
    Source: Source<'source>,
    Self: WithSource<Source>, 

Returns a lookup table for the Lexer

fn lexer<'source, Source>(source: Source) -> Lexer<Self, Source> where
    Self: WithSource<Source>,
    Source: Source<'source>, 

Create a new instance of a Lexer that will produce tokens implementing this Logos.