Enum Token 

pub enum Token {
    Integer(i64),
    Float(f64),
    SingleString(String),
    DoubleString(String),
    BacktickString(String),
    Regex(String, String, char),
    HereDoc(String, String, bool),
    QW(Vec<String>),
    ScalarVar(String),
    DerefScalarVar(String),
    ArrayVar(String),
    HashVar(String),
    ArrayAt,
    HashPercent,
    Ident(String),
    Label(String),
    PackageSep,
    FormatDecl {
        name: String,
        lines: Vec<String>,
    },
    Plus,
    Minus,
    Star,
    Slash,
    Percent,
    Power,
    Dot,
    X,
    NumEq,
    NumNe,
    NumLt,
    NumGt,
    NumLe,
    NumGe,
    Spaceship,
    StrEq,
    StrNe,
    StrLt,
    StrGt,
    StrLe,
    StrGe,
    StrCmp,
    LogAnd,
    LogOr,
    LogNot,
    LogAndWord,
    LogOrWord,
    LogNotWord,
    DefinedOr,
    BitAnd,
    BitOr,
    BitXor,
    BitNot,
    ShiftLeft,
    ShiftRight,
    Assign,
    PlusAssign,
    MinusAssign,
    MulAssign,
    DivAssign,
    ModAssign,
    PowAssign,
    DotAssign,
    AndAssign,
    OrAssign,
    XorAssign,
    ShiftLeftAssign,
    ShiftRightAssign,
    BitAndAssign,
    BitOrAssign,
    DefinedOrAssign,
    Increment,
    Decrement,
    BindMatch,
    BindNotMatch,
    Arrow,
    FatArrow,
    PipeForward,
    ThreadArrow,
    ThreadArrowLast,
    ThreadArrowStream,
    ThreadArrowStreamLast,
    ThreadArrowPar,
    ThreadArrowParLast,
    Range,
    RangeExclusive,
    Backslash,
    LParen,
    RParen,
    LBracket,
    RBracket,
    LBrace,
    RBrace,
    ArrowBrace,
    Semicolon,
    Comma,
    Question,
    Colon,
    Diamond,
    ReadLine(String),
    FileTest(char),
    Eof,
    Newline,
}

Variants

Integer(i64)

Float(f64)

SingleString(String)

DoubleString(String)

BacktickString(String)

`...` or qx{...} — interpolated like double quotes, then executed as sh -c (Perl qx).

Regex(String, String, char)

Regex pattern: (pattern, flags, delimiter)

HereDoc(String, String, bool)

QW(Vec<String>)

ScalarVar(String)

DerefScalarVar(String)

$$foo — symbolic scalar deref (inner name is foo without sigil).

ArrayVar(String)

HashVar(String)

ArrayAt

HashPercent

Ident(String)

Label(String)

PackageSep

FormatDecl

format NAME = … body … . (body lines without the closing .)

Fields

name: String

lines: Vec<String>

Plus

Minus

Star

Slash

Percent

Power

Dot

X

NumEq

NumNe

NumLt

NumGt

NumLe

NumGe

Spaceship

StrEq

StrNe

StrLt

StrGt

StrLe

StrGe

StrCmp

LogAnd

LogOr

LogNot

LogAndWord

LogOrWord

LogNotWord

DefinedOr

BitAnd

BitOr

BitXor

BitNot

ShiftLeft

ShiftRight

Assign

PlusAssign

MinusAssign

MulAssign

DivAssign

ModAssign

PowAssign

DotAssign

AndAssign

OrAssign

XorAssign

ShiftLeftAssign

ShiftRightAssign

BitAndAssign

Bitwise &=

BitOrAssign

Bitwise |=

DefinedOrAssign

Increment

Decrement

BindMatch

BindNotMatch

Arrow

FatArrow

PipeForward

|> — pipe-forward (F#/Elixir): x |> f(a) desugars to f(x, a) at parse time.

ThreadArrow

~> — thread-first macro: ~> EXPR stage1 stage2 ... injects as first arg

ThreadArrowLast

~>> / ->> — thread-last macro: injects as last arg

ThreadArrowStream

~s> — streaming thread-first. Per-stage semantics match ~> (insert threaded value as first arg / topic), but each stage runs in its own worker connected by bounded channels — items flow one at a time. Concurrent (per-item flow with backpressure), not chunk-parallel.

ThreadArrowStreamLast

~s>> — streaming thread-last. Per-stage semantics match ~>> (insert threaded value as last arg).

ThreadArrowPar

~p> — parallel-chunk thread-first. Whole pipeline runs per chunk in parallel, results auto-merged at end (sugar for par_reduce { stage1 |> stage2 |> ... } SOURCE). ||> or |then| switch from parallel-chunk back to pipe-forward / ~>.

ThreadArrowParLast

~p>> — parallel-chunk thread-last counterpart of ~p>.

Range

Two-dot range / inclusive flip-flop (..).

RangeExclusive

Three-dot range / exclusive flip-flop (...); list expansion matches .. (Perl).

Backslash

LParen

RParen

LBracket

RBracket

LBrace

RBrace

ArrowBrace

>{ — standalone block in thread macro (not attached to a function)

Semicolon

Comma

Question

Colon

Diamond

ReadLine(String)

FileTest(char)

Eof

Newline

Implementations

impl Token

pub fn is_term_start(&self) -> bool

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Token