Enum Token 

pub enum Token {
    None,
    Whitespace,
    Semicolon,
    Colon,
    Comma,
    Pipe,
    At,
    Bang,
    Variable(String),
    Curly(Side),
    Square(Side),
    Paren(Side),
    Angle(Side),
    Comment(String),
    Ident(String),
    Num(i64),
    Str(String),
}

A token that we derive from text

Variants

None

An empty token with no meaning. These technically shouldn’t appear in the output, but just skip over them.

Whitespace

Any whitespace, such as tabs and newlines

Semicolon

A semicolon (;)

Colon

A colon (:)

Comma

A comma (,)

Pipe

A pipe / bar (|)

At

An at symbol (@)

Bang

An exclamation point (!)

Variable(String)

A variable ($var_name)

Curly(Side)

A curly brace ({ / })

Square(Side)

A square bracket ([ / ])

Paren(Side)

A parenthese (( / ))

Angle(Side)

An angle bracket (< / >)

Comment(String)

A comment

Ident(String)

An identifier (foo)

Num(i64)

An integer number (-12, 6, 128, etc.)

Str(String)

A string literal (“‘hello’ there”)

Implementations

impl Token

pub fn as_string(&self) -> String

Print this token as a string

pub fn is_ignored(&self) -> bool

Checks if this token is a useless character with no meaning

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Token