Expand description
Leaf token types and the delimiter Group aliases.
The unit and single-field leaves (KwLet, VarTok, IntTok, …) are
generated by token_leaves! in crate::token and re-exported here (so
use crate::leaf::* still names them). This module keeps only what that
macro cannot express — the multi-field qualified-name leaves,
LengthTok/LiteralTok, the multi-variant BinOpTok, and the
AnyHorzCmdTok/AnyVertCmdTok alternations — plus the delimiter Group
aliases, which are the core atom-generic syan::nested::group::Group.
Re-exports§
pub use crate::token::*;
Structs§
- BinOp
Tok - A binary-operator token: the
binopfamily ofnxlor..nxrtimesplus themod/::operators that also act as binops (parser.mly’sbinopnonterminal, minusUNOP_EXCLAM/BEFORE/LNOT, which are not simple infix operators in this grammar’s flattened operator-chain shape). Deliberately excludesToken::Bar(match-arm separator) andToken::ExactAmp(the&-prefixed “next” unary operator) — seecst.rs’s note onBarhandling. - Exact
Times Tok - A standalone
*(EXACT_TIMESinparser.mly) in type-expression position — the product-type separator (type point = length * length,cst.rs’sTypeProd). Hand-written (not#[leaf(...)]-generated onToken::ExactTimesintoken.rs) becauseToken::ExactTimesalready doubles as one ofBinOpTok’s matched variants for the expression-level*; this leaf letsTypeProdconsume the same token without pulling in the rest of the binop set, mirroringLengthTok’s boilerplate. - Header
Stage Tok @stage: persistent/@stage: 0/@stage: 1header token (cst.rs’sHeader::Stage). Hand-written likeBinOpTokabove: the three spellings are separate unitTokenvariants with no shared payload for a#[leaf(...)]derive to key off, so this matches any of them and keeps whichever one matched (needed for a lossless round-trip —Unparsereplays exactly the token that was read).- Horz
CmdWith ModTok - A module-qualified inline command, e.g.
\Mod.cmd. - Length
Tok - A length constant such as
12pt(two payload fields, hand-written). - Literal
Tok - A backtick string literal with its space-trimming flags.
- Long
Upper Tok - A dotted module path ending in an UPPER segment, e.g.
A.B.C(upstreamLONG_UPPER) — module-expression paths, functor application operands, and signature paths (mod_chain,sigexpr_bot). V0_1-only (a lex error under V0_0). - Math
CmdWith ModTok - A module-qualified math command, e.g.
\Mod.cmdin math mode. - Naming
OpTok - The operator token accepted inside a
( ‹op› )NAMING form (seeOpNameTok) —BinOpTok’s whole alternative set, PLUS!/before. Upstreamparser.mly’sbinopnonterminal is used in exactly two productions, both naming forms (VAL LPAREN binop RPARENandLPAREN binop RPARENas a bare atomic-expression reference) — never for infix chaining — and it acceptsUNOP_EXCLAM/BEFORE/LNOTthere, unlike this grammar’s flattened infix operator chain (cst.rs’sOpRhs, which usesBinOpTokdirectly and correctly excludes them:!/beforeare not simple infix operators here).LNOT(not) has no reserved keyword token in this port at all (see thenotnote besidecrate::token::Token::Mod): it lexes as an ordinaryVarTok, solet not x = ../val not : ty(unparenthesized) already parse via the plain-name arm ofcst.rs’sBindName/ast::Atomic:: OpRef’s siblingVararm, with no token to add here. Kept as its own struct (mirroringBinOpTok’s shape) rather than wideningBinOpTokitself, so the infix-chain grammar’s exclusion stays intact by construction — nothing downstream ofBinOpTok::parsecan ever see!/before. - OpName
Tok ( ‹op› )— a parenthesized (possibly user-defined) operator name. Two surface uses share this leaf: a binding-position NAME (cst.rs’sBindName, e.g.let (+++>) = ../val (-->) : ty— the gap blockingitemize.satyh/progsynt.satyh) and a bare atomic-expression reference to an operator as a first-class value (cst.rs’sast::Atomic::OpRef, e.g.(+++))..nameis the operator’s text (NamingOpTok::op_text);.spancovers the whole( .. ), delimiters included. Both are precomputed here (rather than left to theSpannedtrait) so a downstream crate with no directsyandependency (rustyfi-lang) can read them as plain fields, exactly like every other leaf’s.name/.span. Hand-written (not#[leaf(...)]-generated) for the same reason asBinOpTok: it spans three atoms, not one. The inner operator is aNamingOpTok(not a bareBinOpTok) so(!)/(before)— valid only in this naming position, per upstream’sbinopnonterminal — also parse; seeNamingOpTok’s doc comment.- Unit
Paren Unparsefor an EMPTY group used as an ordinary field rather than as the target of a#[group(self.x)]—PatBot::Unit { paren: ParenGroup<()> }and its three siblings, which spell()with nothing inside.- VarIn
Horz Tok #var(or#Mod.var) in inline text, before the mode switches to active.- VarIn
Math Tok #var(or#Mod.var) in math mode.- VarIn
Vert Tok #var(or#Mod.var) in block text, before the mode switches to active.- VarWith
ModTok - A module-qualified variable, e.g.
Mod.x. - Vert
CmdWith ModTok - A module-qualified block command, e.g.
+Mod.cmd.
Enums§
- AnyHorz
CmdTok - Either sigil-only (
\cmd) or module-qualified (\Mod.cmd) inline command name —hcmdinparser.mly. Not itself recursive, so (unlikecrate::cst::ast::InlineElem) it can be a plain top-level derive. - AnyMath
CmdTok - Either sigil-only (
\cmd) or module-qualified (\Mod.cmd) math command name in math mode — the math analogue ofAnyHorzCmdTok. - AnyVert
CmdTok - Either sigil-only (
+cmd) or module-qualified (+Mod.cmd) block command name —vcmdinparser.mly.
Type Aliases§
- Block
Group '< … >/< … >block text.- Inline
Group { … }inline text.- List
Group [ … ]list.- Math
Group ${ … }/{ … }math (the latter when already inside math mode).- Open
Module Group Mod.( … )— the open-module-scope expression (cst.rs’sAtomic::OpenModule). The open delimiter (OpenModuleTok, carrying the module name) isMod.(; the close is a plain)(RParenTok, exactly likeParenGroup’s).- Paren
Group ( … )in program mode.- Record
Group (| … |)record.