Enum pupil::lexer::Token

pub enum Token<'a> {
    Unk(&'a str),
    Lit(Value),
    Op(Operator),
    Var(&'a str),
    Open(&'a str),
    Comma,
    Close,
}

Supported tokens types.

Variants

Unk(&'a str)

Unknown token.

It’s the caller’s responsibility to handle this with an error of some kind.

This will swallow the entire remainder of the input, such that the tokenization finishes on next iteration.

Lit(Value)

Value literal token.

Negative literals are Token::Op(Operator::Sub) followed by a positive literal.

Op(Operator)

Operator token.

Var(&'a str)

Variable token.

Alphanumeric characters only. Not followed by a (.

Open(&'a str)

Function token.

Alphanumeric characters only. Implicitly followed by a (.

Comma

Comma token ,.

Used to provide multiple arguments to a function.

Close

Function closing token ).

Trait Implementations

impl<'a> Clone for Token<'a>

fn clone(&self) -> Token<'a>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a> Debug for Token<'a>

fn fmt(&self, __arg_0: &mut Formatter) -> Result

Formats the value using the given formatter.

impl<'a> PartialEq for Token<'a>

fn eq(&self, __arg_0: &Token<'a>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, __arg_0: &Token<'a>) -> bool

This method tests for !=.