Enum yash_syntax::parser::lex::TokenId

pub enum TokenId {
    Token(Option<Keyword>),
    Operator(Operator),
    IoNumber,
    EndOfInput,
}

Token identifier, or classification of tokens.

This enum classifies a token as defined in POSIX XCU 2.10.1 Shell Grammar Lexical Conventions, but does not exactly reflect further distinction defined in POSIX XCU 2.10.2 Shell Grammar Rules.

For convenience, the special token identifier EndOfInput is included.

Variants

Token(Option<Keyword>)

TOKEN

If this token looks like a reserved word, this variant has some associated Keyword that describes the word. However, it depends on context whether a token is actually regarded as a reserved word or just as an ordinary word. You must ensure that you’re in an applicable context when examining the Keyword value.

Operator(Operator)

Operator

IoNumber

IO_NUMBER

EndOfInput

Imaginary token identifier for the end of input.

Trait Implementations

impl Clone for TokenId

fn clone(&self) -> TokenId

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenId

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq<TokenId> for TokenId

fn eq(&self, other: &TokenId) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &TokenId) -> bool

This method tests for !=.

impl Copy for TokenId

impl Eq for TokenId

impl StructuralEq for TokenId

impl StructuralPartialEq for TokenId