Enum TokenCategory 

pub enum TokenCategory {
    Keyword,
    Operator,
    String,
    Number,
    Variable,
    Comment,
    Punctuation,
    Command,
    Path,
    Flag,
    Error,
}

Semantic category for syntax highlighting.

Stable enum that groups tokens by purpose. Consumers match on categories instead of individual tokens, insulating them from lexer evolution.

Not #[non_exhaustive], deliberately: this is the enum Token (which is #[non_exhaustive]) exists to protect embedders from — it is the small, closed vocabulary a syntax highlighter matches exhaustively so a new Token variant never breaks it. Making this one grow too would defeat the reason it exists.

Variants

Keyword

Keywords: if, then, else, for, while, function, return, etc.

Operator

Operators: |, &&, ||, >, >>, 2>&1, =, ==, etc.

String

String literals: “…”, ‘…’, heredocs

Number

Numeric literals: 123, 3.14, arithmetic expressions

Variable

Variable references: $foo, ${bar}, $1, $@, $#, $?, $$

Comment

Comments: # …

Punctuation

Punctuation: ; , . ( ) { } [ ]

Command

Identifiers in command position

Path

Absolute paths: /foo/bar

Flag

Flags: –long, -s, +x

Error

Invalid tokens

Trait Implementations

impl Clone for TokenCategory

fn clone(&self) -> TokenCategory

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for TokenCategory

impl Debug for TokenCategory

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for TokenCategory

impl Hash for TokenCategory

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for TokenCategory

fn eq(&self, other: &TokenCategory) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for TokenCategory