Enum Token 

pub enum Token {
    Set,
    Local,
    If,
    Then,
    Else,
    Elif,
    Fi,
    For,
    While,
    In,
    Do,
    Done,
    Case,
    Esac,
    Function,
    Break,
    Continue,
    Return,
    Exit,
    True,
    False,
    TypeString,
    TypeInt,
    TypeFloat,
    TypeBool,
    And,
    Or,
    EqEq,
    NotEq,
    Match,
    NotMatch,
    GtEq,
    LtEq,
    GtGt,
    StderrToStdout,
    StdoutToStderr,
    StdoutToStderr2,
    Stderr,
    Both,
    HereString,
    HereDocStart,
    DoubleSemi,
    Eq,
    Pipe,
    Amp,
    Gt,
    Lt,
    Semi,
    Colon,
    Comma,
    DotDot,
    Dot,
    TildePath(String),
    Tilde,
    RelativePath(String),
    DotSlashPath(String),
    DottedIdent(String),
    LBrace,
    RBrace,
    LBracket,
    RBracket,
    LParen,
    RParen,
    Star,
    Bang,
    Question,
    GlobWord(String),
    Arithmetic(String),
    CmdSubstStart,
    LongFlag(String),
    ShortFlag(String),
    PlusFlag(String),
    DoubleDash,
    PlusBare(String),
    MinusBare(String),
    JobSpec(String),
    MinusAlone,
    String(String),
    SingleString(String),
    VarRef(String),
    SimpleVarRef(String),
    Positional(usize),
    AllArgs,
    ArgCount,
    LastExitCode,
    CurrentPid,
    VarLength(String),
    HereDoc(HereDocData),
    Int(i64),
    Float(f64),
    NumberIdent(String),
    InvalidFloatNoLeading,
    InvalidFloatNoTrailing,
    Path(String),
    Ident(String),
    Comment,
    Newline,
    LineContinuation,
    BacktickRejected,
}

Variants

Set

Local

If

Then

Else

Elif

Fi

For

While

In

Do

Done

Case

Esac

Function

Break

Continue

Return

Exit

True

False

TypeString

TypeInt

TypeFloat

TypeBool

And

Or

EqEq

NotEq

Match

NotMatch

GtEq

LtEq

GtGt

StderrToStdout

StdoutToStderr

StdoutToStderr2

Stderr

Both

HereString

HereDocStart

DoubleSemi

Eq

Pipe

Amp

Gt

Lt

Semi

Colon

Comma

DotDot

Dot

TildePath(String)

Tilde path: ~/foo, ~user/bar - value includes the full string

Tilde

Bare tilde: ~ alone (expands to $HOME)

RelativePath(String)

Relative path: ../foo/bar, bare src/kaish (ident containing /), or a directory reference with a trailing slash like dest/. The trailing-slash form uses * (not +) after the slash so dest/ lexes as one token instead of Ident("dest") + Path("/") — the latter split silently turned cp a b dest/ into a 4-operand command.

DotSlashPath(String)

Dot-slash path: ./foo, ./script.sh

DottedIdent(String)

Dot-prefixed bareword: .parent, .gitignore, .foo.bar. Treated as an opaque string in argv position. Distinct from Token::Dot (the POSIX . source alias) which only matches a bare . — the source alias requires whitespace before its file argument (. script), so .parent (no space) is unambiguously a single bareword.

LBrace

RBrace

LBracket

RBracket

LParen

RParen

Star

Bang

Question

GlobWord(String)

Merged glob word: span-adjacent tokens containing *, ?, or [...]. Synthesized by merge_glob_adjacent(), never produced by logos directly.

Arithmetic(String)

Arithmetic expression content: synthesized by preprocessing. Contains the expression string between $(( and )).

CmdSubstStart

Command substitution start: $( - begins a command substitution

LongFlag(String)

Long flag: --name or --foo-bar

ShortFlag(String)

Short flag: -l, -la (combined short flags), or a dash-word with internal hyphens like -not-a-flag. Internal hyphens are part of the single shell word — without them the word fragments into separate flag tokens, which breaks echo -- -not-a-flag and the like. A leading -- is still DoubleDash (the second char must be a letter here), and whether the word is a flag or a literal is the binding layer’s call.

PlusFlag(String)

Plus flag: +e or +x (for set +e to disable options)

DoubleDash

Double dash: -- alone marks end of flags

PlusBare(String)

Bare word starting with + followed by non-letter: +%s, +%Y-%m-%d For date format strings and similar. Lower priority than PlusFlag.

MinusBare(String)

Bare word starting with - followed by non-letter/digit/dash: -%, etc. For rare cases. Lower priority than ShortFlag, Int, and DoubleDash. Excludes - after first - to avoid matching –name patterns.

JobSpec(String)

Job specifier: %1, %2 — the bash idiom for wait/kill targets. Keeps the leading % (kill uses it to distinguish a job from a PID; wait strips it). Without this token a bare %1 is a lexer error.

MinusAlone

Standalone - (stdin indicator for cat -, diff - -, etc.) Only matches when followed by whitespace or end. This is handled specially in the parser as a positional arg.

String(String)

Double-quoted string: "..." - value is the parsed content (quotes removed, escapes processed)

SingleString(String)

Single-quoted string: '...' - literal content, no escape processing

VarRef(String)

Braced variable reference: ${VAR} or ${VAR.field} - value is the raw inner content

SimpleVarRef(String)

Simple variable reference: $NAME - just the identifier

Positional(usize)

Positional parameter: $0 through $9

AllArgs

All positional parameters: $@

ArgCount

Number of positional parameters: $#

LastExitCode

Last exit code: $?

CurrentPid

Current shell PID: $$

VarLength(String)

Variable string length: ${#VAR}

HereDoc(HereDocData)

Here-doc content: synthesized by preprocessing, not directly lexed. Contains the full content of the here-doc (without the delimiter lines).

Int(i64)

Integer literal - value is the parsed i64

Float(f64)

Float literal - value is the parsed f64

NumberIdent(String)

Digit-leading bareword: 019dda1c (SHA prefix), UUIDs, version-ish strings. Distinguished from Int because at least one alpha character follows the leading digits — the lexer commits to “this is a string, not a number.” Treated as a bareword string in expression position.

InvalidFloatNoLeading

Invalid: float without leading digit (like .5)

InvalidFloatNoTrailing

Invalid: float without trailing digit (like 5.) Logos uses longest-match, so valid floats like 5.5 will match Float pattern instead

Path(String)

Absolute path: /tmp/out, /etc/hosts, etc.

Ident(String)

Identifier - value is the identifier string Allows dots for filenames like script.kai

Comment

Comment: # ... to end of line

Newline

Newline (significant in kaish - ends statements)

LineContinuation

Line continuation: backslash at end of line

BacktickRejected

Backtick command substitution — explicitly rejected. Kaish drops backticks; the callback always errors so users get a dedicated BackticksNotSupported message instead of the generic UnexpectedCharacter they would have hit before. Backticks inside single/double-quoted strings, heredoc bodies, and comments don’t reach this match — those tokens are matched as a single unit (strings) or extracted before logos runs (heredocs) or skipped to EOL (comments).

Implementations

impl Token

pub fn category(&self) -> TokenCategory

Returns the semantic category for syntax highlighting.

impl Token

pub fn is_keyword(&self) -> bool

Returns true if this token is a keyword.

pub fn is_type(&self) -> bool

Returns true if this token is a type keyword.

pub fn starts_statement(&self) -> bool

Returns true if this token starts a statement.

pub fn is_value(&self) -> bool

Returns true if this token can appear in an expression.

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'s> Logos<'s> for Token

type Error = LexerError

Error type returned by the lexer. This can be set using #[logos(error = MyError)]. Defaults to () if not set.

type Extras = ()

Associated type Extras for the particular lexer. This can be set using #[logos(extras = MyExtras)] and accessed inside callbacks.

type Source = str

Source type this token can be lexed from. This will default to str, unless one of the defined patterns explicitly uses non-unicode byte values or byte slices, in which case that implementation will use [u8].

fn lex( lex: &mut Lexer<'s, Self>, ) -> Option<Result<Self, <Self as Logos<'s>>::Error>>

The heart of Logos. Called by the Lexer. The implementation for this function is generated by the logos-derive crate.

fn lexer(source: &'source Self::Source) -> Lexer<'source, Self>

where Self::Extras: Default,

Create a new instance of a Lexer that will produce tokens implementing this Logos.

fn lexer_with_extras( source: &'source Self::Source, extras: Self::Extras, ) -> Lexer<'source, Self>

Create a new instance of a Lexer with the provided Extras that will produce tokens implementing this Logos.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Token