Enum Token 

pub enum Token {
    Comment,
    Constant,
    Delim,
    Escape,
    Field,
    Key,
    ModuleName,
    ModuleItem,
    Keyword,
    Literal,
    Number,
    Operator,
    StringDelim,
    Variable,
    Sigil,
}

Syntactic token

Variants

Comment

Comment

Constant

Non-numeric, non-string constant such as nil or false

Delim

Delimiter such as (, [, etc

Escape

Escape within a string

Field

Field, e.g. bar in foo.bar

Key

Key such as foo:

ModuleName

Module name

ModuleItem

Module item

Keyword

Keyword such as while, for, do, import

Literal

Literal string, including non-escape, non-interpolated portions of quoted strings

Number

Numeric constant

Operator

Unary or binary operator such as + or -

StringDelim

A string delimeter (")

Variable

A variable

Sigil

A sigil like $ or ...

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Copy for Token