Enum TokenKind 

pub enum TokenKind {
    Eof,
    Ident(String),
    Keyword(Keyword),
    Int(IntLit),
    Float(FloatLit),
    Char(CharLit),
    Str(StrLit),
    Punct(Punct),
    Error(String),
}

What a Token is.

Variants

Eof

End of the token list. Always present, exactly once, last.

Ident(String)

An identifier that is not a keyword.

Keyword(Keyword)

A keyword.

Int(IntLit)

An integer constant.

Float(FloatLit)

A floating constant.

Char(CharLit)

A character constant.

Str(StrLit)

A string literal.

Punct(Punct)

A punctuator.

Error(String)

Text that is not a C token at all, kept so that a skipped group may contain it. Holds its own spelling.

Implementations

impl TokenKind

pub fn describe(&self) -> String

A short description used in “expected …, found …” messages.

pub fn spelling(&self) -> &str

The exact source spelling of this token.

This is what # stringifies and what ## pastes, so it has to be the text as written — 0x1f rather than 31, '\n' rather than a newline. TokenKind::Eof has no spelling and gives "".

pub fn macro_name(&self) -> Option<&str>

The name this token has when it is used as a macro name.

Keywords are ordinary identifiers during translation phase 4 — our lexer classifies them early, so #define restrict and #ifdef inline would otherwise be unusable — which is why a keyword answers with its spelling here.

Trait Implementations

impl Clone for TokenKind

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenKind

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for TokenKind

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for TokenKind