Enum Token 

pub enum Token {
    Integer(i64),
    Float(f64),
    SingleString(String),
    DoubleString(String),
    BacktickString(String),
    Regex(String, String, char),
    HereDoc(String, String, bool),
    QW(Vec<String>),
    ScalarVar(String),
    DerefScalarVar(String),
    ArrayVar(String),
    HashVar(String),
    ArrayAt,
    HashPercent,
    Ident(String),
    Label(String),
    PackageSep,
    FormatDecl {
        name: String,
        lines: Vec<String>,
    },
    Plus,
    Minus,
    Star,
    Slash,
    Percent,
    Power,
    Dot,
    X,
    NumEq,
    NumNe,
    NumLt,
    NumGt,
    NumLe,
    NumGe,
    Spaceship,
    StrEq,
    StrNe,
    StrLt,
    StrGt,
    StrLe,
    StrGe,
    StrCmp,
    LogAnd,
    LogOr,
    LogNot,
    LogAndWord,
    LogOrWord,
    LogNotWord,
    DefinedOr,
    BitAnd,
    BitOr,
    BitXor,
    BitNot,
    ShiftLeft,
    ShiftRight,
    Assign,
    PlusAssign,
    MinusAssign,
    MulAssign,
    DivAssign,
    ModAssign,
    PowAssign,
    DotAssign,
    XAssign,
    AndAssign,
    OrAssign,
    XorAssign,
    ShiftLeftAssign,
    ShiftRightAssign,
    BitAndAssign,
    BitOrAssign,
    DefinedOrAssign,
    Increment,
    Decrement,
    BindMatch,
    BindNotMatch,
    Arrow,
    FatArrow,
    PipeForward,
    ThreadArrow,
    ThreadArrowLast,
    ThreadArrowStream,
    ThreadArrowStreamLast,
    ThreadArrowPar,
    ThreadArrowParLast,
    ThreadArrowDist,
    ThreadArrowDistLast,
    Range,
    RangeExclusive,
    Backslash,
    LParen,
    RParen,
    LBracket,
    RBracket,
    LBrace,
    RBrace,
    ArrowBrace,
    Semicolon,
    Comma,
    Question,
    Colon,
    Diamond,
    ReadLine(String),
    FileTest(char),
    Eof,
    Newline,
}

Token — see variants.

Variants

Integer(i64)

Integer variant.

Float(f64)

Float variant.

SingleString(String)

SingleString variant.

DoubleString(String)

DoubleString variant.

BacktickString(String)

`...` or qx{...} — interpolated like double quotes, then executed as sh -c (Perl qx).

Regex(String, String, char)

Regex pattern: (pattern, flags, delimiter)

HereDoc(String, String, bool)

HereDoc variant.

QW(Vec<String>)

QW variant.

ScalarVar(String)

ScalarVar variant.

DerefScalarVar(String)

$$foo — symbolic scalar deref (inner name is foo without sigil).

ArrayVar(String)

ArrayVar variant.

HashVar(String)

HashVar variant.

ArrayAt

ArrayAt variant.

HashPercent

HashPercent variant.

Ident(String)

Ident variant.

Label(String)

Label variant.

PackageSep

PackageSep variant.

FormatDecl

format NAME = … body … . (body lines without the closing .)

Fields

name: String

lines: Vec<String>

Plus

Plus variant.

Minus

Minus variant.

Star

Star variant.

Slash

Slash variant.

Percent

Percent variant.

Power

Power variant.

Dot

Dot variant.

X

X variant.

NumEq

NumEq variant.

NumNe

NumNe variant.

NumLt

NumLt variant.

NumGt

NumGt variant.

NumLe

NumLe variant.

NumGe

NumGe variant.

Spaceship

Spaceship variant.

StrEq

StrEq variant.

StrNe

StrNe variant.

StrLt

StrLt variant.

StrGt

StrGt variant.

StrLe

StrLe variant.

StrGe

StrGe variant.

StrCmp

StrCmp variant.

LogAnd

LogAnd variant.

LogOr

LogOr variant.

LogNot

LogNot variant.

LogAndWord

LogAndWord variant.

LogOrWord

LogOrWord variant.

LogNotWord

LogNotWord variant.

DefinedOr

DefinedOr variant.

BitAnd

BitAnd variant.

BitOr

BitOr variant.

BitXor

BitXor variant.

BitNot

BitNot variant.

ShiftLeft

ShiftLeft variant.

ShiftRight

ShiftRight variant.

Assign

Assign variant.

PlusAssign

PlusAssign variant.

MinusAssign

MinusAssign variant.

MulAssign

MulAssign variant.

DivAssign

DivAssign variant.

ModAssign

ModAssign variant.

PowAssign

PowAssign variant.

DotAssign

DotAssign variant.

XAssign

x= — string-repetition compound assign ($s x= 3).

AndAssign

AndAssign variant.

OrAssign

OrAssign variant.

XorAssign

XorAssign variant.

ShiftLeftAssign

ShiftLeftAssign variant.

ShiftRightAssign

ShiftRightAssign variant.

BitAndAssign

Bitwise &=

BitOrAssign

Bitwise |=

DefinedOrAssign

DefinedOrAssign variant.

Increment

Increment variant.

Decrement

Decrement variant.

BindMatch

BindMatch variant.

BindNotMatch

BindNotMatch variant.

Arrow

Arrow variant.

FatArrow

FatArrow variant.

PipeForward

|> — pipe-forward (F#/Elixir): x |> f(a) desugars to f(x, a) at parse time.

ThreadArrow

~> — thread-first macro: ~> EXPR stage1 stage2 ... injects as first arg

ThreadArrowLast

~>> / ->> — thread-last macro: injects as last arg

ThreadArrowStream

~s> — streaming thread-first. Per-stage semantics match ~> (insert threaded value as first arg / topic), but each stage runs in its own worker connected by bounded channels — items flow one at a time. Concurrent (per-item flow with backpressure), not chunk-parallel.

ThreadArrowStreamLast

~s>> — streaming thread-last. Per-stage semantics match ~>> (insert threaded value as last arg).

ThreadArrowPar

~p> — parallel-chunk thread-first. Whole pipeline runs per chunk in parallel, results auto-merged at end (sugar for par_reduce { stage1 |> stage2 |> ... } SOURCE). ||> or |then| switch from parallel-chunk back to pipe-forward / ~>.

ThreadArrowParLast

~p>> — parallel-chunk thread-last counterpart of ~p>.

ThreadArrowDist

~d> — distributed thread-first. Same chunk-block semantics as ~p> (each stage operates on @_ = chunk elements), but the chunks are shipped to remote workers on a cluster instead of local rayon threads. Syntax: ~d> on $cluster SOURCE stage1 stage2 .... Sugar for dist_reduce on $cluster { stages } SOURCE. Reuses the existing pmap_on dispatcher (one ssh process per slot, JOB frames flowing over a shared work queue, fault tolerance via retry).

ThreadArrowDistLast

~d>> — distributed thread-last counterpart of ~d> (insert threaded value as last positional arg to each named stage).

Range

Two-dot range / inclusive flip-flop (..).

RangeExclusive

Three-dot range / exclusive flip-flop (...); list expansion matches .. (Perl).

Backslash

Backslash variant.

LParen

LParen variant.

RParen

RParen variant.

LBracket

LBracket variant.

RBracket

RBracket variant.

LBrace

LBrace variant.

RBrace

RBrace variant.

ArrowBrace

>{ — standalone block in thread macro (not attached to a function)

Semicolon

Comma

Question

Colon

Diamond

ReadLine(String)

FileTest(char)

Eof

Newline

Implementations

impl Token

pub fn is_term_start(&self) -> bool

is_term_start — see implementation.

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Token