Enum yash_syntax::parser::lex::TokenId

pub enum TokenId {
    Token(Option<Keyword>),
    Operator(Operator),
    IoNumber,
    EndOfInput,
}

Token identifier, or classification of tokens.

This enum classifies a token as defined in POSIX XCU 2.10.1 Shell Grammar Lexical Conventions, but does not exactly reflect further distinction defined in POSIX XCU 2.10.2 Shell Grammar Rules.

For convenience, the special token identifier EndOfInput is included.

Variants

Token(Option<Keyword>)

TOKEN

If this token looks like a reserved word, this variant has some associated Keyword that describes the word. However, it depends on context whether a token is actually regarded as a reserved word or just as an ordinary word. You must ensure that you’re in an applicable context when examining the Keyword value.

Operator(Operator)

Operator

IoNumber

IO_NUMBER

EndOfInput

Imaginary token identifier for the end of input.

Implementations

impl TokenId

pub fn is_clause_delimiter(self) -> bool

Determines if this token can be a delimiter of a clause.

This function delegates to Keyword::is_clause_delimiter if the token ID is a (possible) keyword, or to Operator::is_clause_delimiter if it is an operator. For EndOfInput the function returns true. Otherwise, the result is false.

Trait Implementations

impl Clone for TokenId

fn clone(&self) -> TokenId

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenId

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for TokenId

fn eq(&self, other: &TokenId) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for TokenId

impl Eq for TokenId

impl StructuralEq for TokenId

impl StructuralPartialEq for TokenId