Struct Token 

pub struct Token {
    pub kind: TokenKind,
    pub range: SourceRange,
    pub bol: bool,
    pub preceded_by_space: bool,
    pub errors: Vec<Diagnostic>,
}

A lexed C token.

Fields

kind: TokenKind

What the token is.

range: SourceRange

Where it is.

bol: bool

Whether it is the first token on its logical line (needed by the preprocessor to spot directives).

preceded_by_space: bool

Whether whitespace or a comment preceded it (needed by the preprocessor’s stringification and macro replacement).

errors: Vec<Diagnostic>

Everything wrong with this token, and with the text between it and the token before it.

The preprocessor reports what is wrong with the token itself if and only if the token reaches its output. What stands whatever becomes of the token — its spelling, and the comments that preceded it — is reported as soon as the token is read from the file, which is what keeps it from being lost when the token opens a directive, names a macro or is an argument the macro drops; see Diagnostic::lexical.

Implementations

impl Token

pub fn keyword(&self) -> Option<Keyword>

The keyword this token is, if any.

pub fn is_punct(&self, p: Punct) -> bool

Whether this token is the given punctuator.

pub fn is_keyword(&self, k: Keyword) -> bool

Whether this token is the given keyword.

pub fn is_eof(&self) -> bool

Whether this token ends the input.

pub fn ident(&self) -> Option<&str>

The identifier this token is, if any.

Trait Implementations

impl Clone for Token

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for Token