Enum Kind 

#[repr(u8)]
pub enum Kind {
    Identifier = 0,
    QuotedIdentifier = 1,
    Keyword = 2,
    Number = 3,
    String = 4,
    Operator = 5,
    Terminator = 6,
    EndOfInput = 7,
}

What a token is.

DuckDB has one IDENTIFIER type covering both a bare word and a quoted one, and recovers the difference by looking at the first byte where it matters. We split them, because the grammar has a QuotedIdentifier rule and asking the token is cheaper and harder to get wrong than asking the source text. The two are otherwise treated alike everywhere upstream treats them alike, which is nearly everywhere.

Variants

Identifier = 0

A bare word that is in no keyword class. foo, and also ascending, which is spelled by a rule and is in no class, so it is a perfectly good column name.

QuotedIdentifier = 1

A word in double quotes. Still an identifier, never a keyword, and case preserving in exactly the same way a bare one is.

Keyword = 2

A bare word that is in at least one keyword class. keyword on this token says which.

Number = 3

A numeric literal, with its underscores, its exponent and its decimal point as written.

String = 4

A string literal, with its quotes. Single quoted, dollar quoted, or one of the four prefixed forms, all of which arrive here as one token.

Operator = 5

Punctuation or an operator run. (, ,, ::, !~~* and + are all this.

Terminator = 6

A ;. Its own kind because Program <- TopLevelStatement* consumes it as one, and because a statement boundary that the tokenizer decided would be a statement boundary the grammar cannot change.

EndOfInput = 7

The sentinel at the end. Always present, always last, always exactly one.

Implementations

impl Kind

pub const fn is_identifier(self) -> bool

Whether this is a name, either spelling.

Trait Implementations

impl Clone for Kind

fn clone(&self) -> Kind

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for Kind

impl Debug for Kind

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for Kind

impl Hash for Kind

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for Kind

fn eq(&self, other: &Kind) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for Kind