Module lexer 

Lossless lexer for LaTeX surface syntax.

The lexer recognizes only bounded modes whose delimiters are visible in the source, including verbatim regions and explicit catcode toggles.

Structs

LexConfig

The lexer’s per-parse mode. flavor fixes the initial catcode regime (a .sty/.cls starts under an implicit \makeatletter), while dtx is an orthogonal axis: when set, the lexer runs the bounded line-oriented docstrip mode for a .dtx file — line-leading % margins become DOC_MARGIN trivia, line-leading %<…> guards become GUARD trivia, and macrocode bodies lex as ordinary code (AGENTS.md decision #1). The two axes are independent because a .dtx’s catcode regime varies by layer (its documentation is Document-flavored, its macrocode Package-flavored), so dtx cannot be folded into a LatexFlavor variant.

ParseCtx

Per-parse facts discovered from definitions or project declarations.

Token

A single lexed token: its kind plus the exact source slice it covers.

Enums

ExplToggle

An expl3 lexer-mode transition.

LatexFlavor

The LaTeX file flavor, fixing the lexer’s initial catcode regime. A document (.tex) starts in the ordinary regime; a package or class (.sty/.cls) is loaded under an implicit \makeatletter, so @ is a letter from the first byte. A trailing explicit \makeatother still applies.

Functions

expl_toggle

Returns the expl3 transition named by a control word.

is_control_word_name

Returns whether name can be a control word in any supported mode.

is_word_char

Returns whether c belongs to an ordinary text token.

lex

Lexes a document into a lossless token stream.

lex_with

Lexes with per-parse facts and an explicit file configuration.

Type Aliases

VerbCtx

Backward-compatible alias for ParseCtx.