#[repr(u16)]
pub enum Token {
Show 58 variants IDENT_KW, DEFINED, DIGITS, PERIOD, PLUS, DASH, SLASH, ASTERISK, PERCENT, LANGLE, RANGLE, LBRACKET, RBRACKET, LPAREN, RPAREN, LBRACE, RBRACE, CARET, BAR, AMPERSAND, TILDE, EQUAL, BANG, COLON, SEMICOLON, COMMA, QUESTION, HASH, QUOTE_STRING, ANGLE_STRING, BACKSLASH, WS, NEWLINE, COMMENT, LINECONT, ERROR, LEFT_OP, RIGHT_OP, INC_OP, DEC_OP, LE_OP, GE_OP, EQ_OP, NE_OP, AND_OP, OR_OP, XOR_OP, MUL_ASSIGN, DIV_ASSIGN, ADD_ASSIGN, MOD_ASSIGN, LEFT_ASSIGN, RIGHT_ASSIGN, AND_ASSIGN, XOR_ASSIGN, OR_ASSIGN, SUB_ASSIGN, PP_CONCAT,
}

Variants

IDENT_KW

Identifier or keyword

DEFINED

defined preprocessor keyword

DIGITS

Digit sequence

PERIOD

.

PLUS

DASH

SLASH

/

ASTERISK

PERCENT

%

LANGLE

<

RANGLE

LBRACKET

[

RBRACKET

]

LPAREN

(

RPAREN

)

LBRACE

{

RBRACE

}

CARET

^

BAR

|

AMPERSAND

&

TILDE

~

EQUAL

=

BANG

!

COLON

:

SEMICOLON

;

COMMA

,

QUESTION

?

HASH

QUOTE_STRING

“string”

ANGLE_STRING

BACKSLASH

\

WS

Whitespace

NEWLINE

Newline

COMMENT

Comment (single-line or multi-line)

LINECONT

Line continuation (required for tracking exact offsets)

ERROR

Invalid token

LEFT_OP

<<

RIGHT_OP

INC_OP

++

DEC_OP

LE_OP

<=

GE_OP

=

EQ_OP

==

NE_OP

!=

AND_OP

&&

OR_OP

||

XOR_OP

^^

MUL_ASSIGN

*=

DIV_ASSIGN

/=

ADD_ASSIGN

+=

MOD_ASSIGN

%=

LEFT_ASSIGN

<<=

RIGHT_ASSIGN

=

AND_ASSIGN

&=

XOR_ASSIGN

^=

OR_ASSIGN

|=

SUB_ASSIGN

-=

PP_CONCAT

Implementations

Trait Implementations

Returns a copy of the value. Read more

Performs copy-assignment from source. Read more

Formats the value using the given formatter. Read more

This method tests for self and other values to be equal, and is used by ==. Read more

This method tests for !=.

Auto Trait Implementations

Blanket Implementations

Gets the TypeId of self. Read more

Immutably borrows from an owned value. Read more

Mutably borrows from an owned value. Read more

Performs the conversion.

Performs the conversion.

The resulting type after obtaining ownership.

Creates owned data from borrowed data, usually by cloning. Read more

🔬 This is a nightly-only experimental API. (toowned_clone_into)

Uses borrowed data to replace owned data, usually by cloning. Read more

The type returned in the event of a conversion error.

Performs the conversion.

The type returned in the event of a conversion error.

Performs the conversion.