Enum TaggedToken

pub enum TaggedToken<T> {
    Import(T),
    As(T),
    Class(T),
    Colon(T),
    Eq(T),
    Comma(T),
    Dot(T),
    Newline(T),
    Null(T),
    Identifier(T, Identifier),
    IntegerLiteral(T, u64),
    Shl(T),
    Mul(T),
    OpenBracket(T),
    CloseBracket(T),
    OpenParen(T),
    CloseParen(T),
    Indent(T),
    Deindent(T),
    DocString(T, String),
}

Token with a tag.

The tag can be used for something like a span location in the original source or to assign an identifier across structures.

Variants

Import(T)

import keyword.

As(T)

as keyword.

Class(T)

class keyword.

Colon(T)

: keyword.

Eq(T)

= keyword.

Comma(T)

, keyword.

Dot(T)

. keyword.

Newline(T)

\n newline.

Null(T)

null keyword.

Identifier(T, Identifier)

An identifier.

IntegerLiteral(T, u64)

An integer literal.

Shl(T)

<< operator.

Mul(T)

* operator.

OpenBracket(T)

[ open bracket.

CloseBracket(T)

] close bracket.

OpenParen(T)

( open parenthesis.

CloseParen(T)

) close parenthesis.

Indent(T)

indent token.

Deindent(T)

deindent token.

DocString(T, String)

Triple double quote docstring.

Implementations

impl<T> TaggedToken<T>

pub fn tag(&self) -> &T

Returns the tag on the token.

pub fn to_untagged(&self) -> Token

Converts the token to a untagged token.

Trait Implementations

impl<T: Clone> Clone for TaggedToken<T>

fn clone(&self) -> TaggedToken<T>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T: Debug> Debug for TaggedToken<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T: PartialEq> PartialEq for TaggedToken<T>

fn eq(&self, other: &TaggedToken<T>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<T: Eq> Eq for TaggedToken<T>

impl<T> StructuralPartialEq for TaggedToken<T>