Struct yash_syntax::parser::lex::Token

pub struct Token {
    pub word: Word,
    pub id: TokenId,
    pub index: usize,
}

Result of lexical analysis produced by the Lexer.

Fields

word: Word

Content of the token.

The word value always contains at least one unit, regardless of whether the token is an operator.

id: TokenId

Token identifier.

index: usize

Position of the first character of the word.

Trait Implementations

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.