Enum TokenTree 

pub enum TokenTree {
    Eoc(EocToken),
    Group(TokenGroup),
    Literal(LiteralToken),
    Punct(Punct),
    Ident(IdentToken),
    Trivia(Trivia),
}

One raw M2 token tree.

Variants

Eoc(EocToken)

A zero-width top-level cell boundary inserted by the lexer.

Group(TokenGroup)

A delimited group.

Literal(LiteralToken)

A literal.

Punct(Punct)

Punctuation.

Ident(IdentToken)

An identifier or keyword spelling.

Trivia(Trivia)

Retained whitespace or a comment.

Implementations

impl TokenTree

pub fn spelling(&self) -> Option<&str>

Returns the token’s source spelling when it is a flat lexical atom.

Examples found in repository

examples/parse_emit_walkthrough.rs (line 23)

    fn parse(input: &mut ParseStream) -> Result<Self, TokenParseError> {
        let Some(token) = input.next_token() else {
            return Err(TokenParseError::UnexpectedEnd {
                expected: "identifier",
                span: Span::detached(),
            });
        };

        let span = token.span();
        match token {
            TokenTree::Ident(identifier) => Ok(Self(identifier)),
            token => Err(TokenParseError::UnexpectedToken {
                expected: "identifier",
                found: token.spelling().unwrap_or("group or trivia").to_owned(),
                span,
            }),
        }
    }

Trait Implementations

impl Clone for TokenTree

fn clone(&self) -> TokenTree

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenTree

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for TokenTree

impl FromIterator<TokenTree> for TokenStream

fn from_iter<T: IntoIterator<Item = TokenTree>>(iter: T) -> Self

Creates a value from an iterator.

impl Lex for TokenTree

fn to_tokens(&self, output: &mut TokenStream)

Appends this value’s token trees to output.

fn to_token_stream(&self) -> TokenStream

Emits this value into a fresh token stream.

fn to_m2(&self) -> String

Renders this value as normalized Macaulay2 source.

impl Parse for TokenTree

fn parse(input: &mut ParseStream) -> Result<Self, TokenParseError>

Consumes this value from input.

impl PartialEq for TokenTree

fn eq(&self, other: &TokenTree) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Spanned for TokenTree

fn span(&self) -> Span

Returns the smallest span covering this value.

impl StructuralPartialEq for TokenTree