Enum TokenKind 

pub enum TokenKind {
    Keyword(Keyword),
    Algorithm(Algorithm),
    Language(Language),
    Distance(DistanceKind),
    VectorType(VectorTypeKind),
    Operator(Operator),
    OpenDelim(Delim),
    CloseDelim(Delim),
    String(StringKind),
    Parameter,
    Identifier,
    LeftChefron,
    RightChefron,
    Star,
    Question,
    Dollar,
    ArrowRight,
    ForwardSlash,
    Dot,
    DotDot,
    DotDotDot,
    SemiColon,
    PathSeperator,
    Colon,
    Comma,
    Vert,
    At,
    Eof,
    Digits,
    NaN,
    Infinity,
    Invalid,
}

The type of token

Variants

Keyword(Keyword)

Algorithm(Algorithm)

Language(Language)

Distance(DistanceKind)

VectorType(VectorTypeKind)

Operator(Operator)

OpenDelim(Delim)

CloseDelim(Delim)

String(StringKind)

a token denoting the opening of a string, i.e. r"

Parameter

A parameter like $name.

Identifier

LeftChefron

<

RightChefron

>

Star

*

Question

?

Dollar

$

ArrowRight

->

ForwardSlash

‘/’

Dot

.

DotDot

..

DotDotDot

... or …

SemiColon

;

PathSeperator

::

Colon

:

Comma

,

Vert

|

At

@

Eof

A token which indicates the end of the file.

Digits

A token consiting of one or more ascii digits.

NaN

The Not-A-Number number token.

Infinity

The infinity number token.

Invalid

A token which could not be properly lexed.

Implementations

impl TokenKind

pub fn has_data(&self) -> bool

pub fn as_str(&self) -> &'static str

Trait Implementations

impl Clone for TokenKind

fn clone(&self) -> TokenKind

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenKind

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for TokenKind

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for TokenKind

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for TokenKind

fn eq(&self, other: &TokenKind) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for TokenKind

impl Eq for TokenKind

impl StructuralPartialEq for TokenKind