████████╗███████╗██╗ ██╗██████╗ ███████╗
╚══██╔══╝██╔════╝╚██╗██╔╝██╔══██╗██╔════╝
██║ █████╗ ╚███╔╝ ██████╔╝███████╗
██║ ██╔══╝ ██╔██╗ ██╔══██╗╚════██║
██║ ███████╗██╔╝ ██╗██║ ██║███████║
╚═╝ ╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝╚══════╝
[TEX'S MOUTH AND EXPANDER, COMPILED TO BYTECODE — NOT INTERPRETED]
"Every TeX since 1982 interprets the expander. This one compiles it."
A TeX engine in Rust: Knuth's mouth and expander, lowered onto
fusevm bytecode and run on the
shared three-tier Cranelift JIT — the same engine every other fusevm frontend
runs on.
Table of Contents
- [0x00] What it is
- [0x01] Install
- [0x02] Usage
- [0x03] What works
- [0x04] Intercepts
- [0x05] Inline Rust
- [0x06] What does not
- [0x07] How it runs
- [0x08] Parity
- [0x09] Fuzzing
- [0x0A] Benchmarks
- [0x0B] Releasing
- [0x0C] Documentation
- [0xFF] Licence
A TeX engine in Rust: Knuth's mouth and expander lowered onto a bytecode VM, and a stomach that typesets to DVI and PDF.
[0x00] What it is
TeX is three machines. The mouth turns bytes into tokens under a mutable
category-code table; the expander turns tokens into other tokens — \def,
\csname, \the, the conditionals; the stomach builds boxes out of what is
left and ships them.
texrs compiles the first two to bytecode. That is the half a macro-heavy
document spends its time in, and the half where a compiled implementation has
something to prove: every mainstream engine (pdfTeX, XeTeX, LuaTeX) descends
from tex.web through web2c and interprets the expander.
The stomach is there too. Both outputs break a paragraph the way tex.web
§813 does — minimising the total demerits of the whole paragraph over every
feasible set of breakpoints, with Liang hyphenation to widen the places a line
may end. --pdf writes the PDF itself; --dvi ships a box tree through
hlist_out/vlist_out (§619-§640), so hpack sets each line's glue and the
file carries it at the width it was set to. --pdf breaks its PAGES the same
way, by LaTeX's widow, orphan, broken-line and heading penalties over the whole
document. Maths is set from tex.web's own mlist_to_hlist, the boxes, glue
setting and page builder underneath are ported, and the .tfm's ligature and
kern program is consulted — so tex's fi ligature is texrs's too. [0x06] says
what is left.
[0x01] Install
# Homebrew (macOS + Linux)
# Or via crates.io
# Or from source
&& &&
Zsh tab completion
Editors
texrs --lsp and texrs --dap are ordinary LSP/DAP servers over stdio, so any
client can drive them. editors/ carries ready-made configuration:
editors/emacs/ is the one that is more than configuration: texrs-mode.el is
a major mode, and texrs-stdlib.el beside it is generated from the same corpus
as docs/reference.html — cargo run --bin gen-emacs-stdlib, gated by
tests/emacs_stdlib.rs, so completion and hover there cannot describe a
primitive the engine does not carry.
Man pages
[0x02] Usage
texrs takes tex's own command line, in all three of its invocation forms:
A bare name gets .tex appended, so texrs doc and texrs doc.tex are the
same run. Options may be spelled with one dash or two, and a value may follow
an = or a space — -jobname=x, --jobname=x and -jobname x all agree.
tex's options:
-interaction=MODE
-jobname=NAME
-output-directory=DIR
-progname=NAME
-fmt=NAME
texrs's own:
Files found through kpsewhich (fonts, hyphenation patterns, the glyph list,
packages) are resolved once per run, by one kpsewhich -interactive process that
reads the TeX databases once and answers every name after, and the paths are
remembered in <cache dir>/texrs/kpse-*.tsv (disabled by TEXRS_CACHE=0). That
file is stamped with the size and modification time of every ls-R database and
texmf.cnf kpathsea reads, and is started over when the stamp no longer matches,
as after a TeX Live update. A remembered path is used only while the file still
exists and no file of that name sits in the working directory.
-X is the second half of the command line: the document commands, and a reader
for each binary format a TeX installation is made of. Every one of them prints
what it read rather than acting on it, which is what makes them usable to find
out why a build is wrong.
A document is a Texrs.toml naming its inputs and one or more outputs. The
output type is messages, tokens or disasm — the \message stream, the
mouth's token stream, or the lowered bytecode — and -X build --profile picks
one by NAME, so a document may carry several of the same kind. A page is not
among them: --pdf and --dvi are flags on a file, not profiles on a document.
texrs(1) has the whole schema, including preamble (files dumped as a format
so a macro file compiles once) and bundle (a zip searched after the
document's own directory).
The environment it reads is on the generated reference page in full. One of
them is worth knowing before you compare two files: a PDF carries the time it
was written, so two runs of one document differ in their dates and nothing
else. SOURCE_DATE_EPOCH (reproducible-builds.org's specification, which
LuaTeX honours too) replaces the wall clock with a fixed time, and with it set
the same document twice gives the same bytes:
&&
&&
Two places the grammar departs from tex, both because texrs takes several
files where tex takes one: a non-option argument is a FILE unless it begins
with \, and options are recognised anywhere rather than only before the
first file, so texrs doc -halt-on-error sets the flag instead of typesetting
it.
Output is written the way tex writes it on the terminal, which is what the parity harnesses compare:
$ texrs examples/macros.tex
(./examples/macros.tex HELLO-WORLD [1|2] )
examples/ carries a runnable program per construct — macros with delimited
parameters, count arithmetic, conditionals, groups, \csname, \edef — and
tests/examples.rs holds every one of them in parity with real tex, with no
known-gap escape hatch. Documentation that has drifted from the engine is worse
than none.
Embedding
run_program
Runs src as a whole document and returns its text, as texrs::run_text
returns it and texrs --text prints it (the \message stream is not part of
it). input is the terminal: each line answers one \read16, \read-1 or
\read from a closed stream (tex.web §484), and the run starts in scroll mode
so those reads are allowed; running out of lines is tex's
*** (job aborted, no legal \end found). Each call runs on its own thread with
a 64 MiB stack and fresh thread-local state, so calls share nothing and
concurrent calls never mix output. A refused document (an undefined control
sequence, a fatal error) or an engine panic is Err; nothing is written to the
process stdout.
let src = "\\catcode`\\{=1 \\catcode`\\}=2\n\\read16 to \\name\nHello, \\name!\n\\end\n";
let out = run_program?;
assert_eq!;
[0x03] What works
- Category codes,
\catcode, and INITEX's sparse defaults —{is not a group character until something makes it one, exactly as a baretexrun behaves. - The three-state line scanner: blank line to
\par, spaces collapsed, the space after a control word swallowed and after a control symbol kept. ^^Xnotation.\defwith undelimited and delimited parameters (\def\pair#1,#2.{...}),##, and nested definitions.\csname/\endcsname,\string,\the,\number,\romannumeral,\jobname,\expandafter,\inputlineno,\show(the meaning and the error context tex prints after it) and\showthe(the value, the same way).\let,\edef/\xdef,\gdef,\global,\begingroup/\endgroup.- Conditionals:
\iftrue,\iffalse,\ifnum,\ifodd,\ifx,\if,\ifcat,\ifcasewith\or,\else,\fi— nested, and inside a\messagebody. - Every primitive luatex has is a primitive here too, with the command code
luatex gives it (
src/luatex_primitives.rs, written from luatex's own table), so\meaning,\ifdefinedand\ifcsnameanswer for\numexpr,\attributeor\Umathcharas luatex does, whether or not texrs executes it;BUGS.mdhas the census of which ones it does.\ifdefinedof a character that is not active is true, as in luatex. \detokenize,\unexpandedand\expandedread their text as e-TeX'sscan_general_textdoes, expanding up to the brace (\detokenize\expandafter{\x}), in running text, in an\edefbody and in a\message;\expanded's result is expanded again where it lands, so a\noexpandin it protects one pass only.\numexprand\dimexpras luatex'sscan_exprevaluates them -- precedence, parentheses, a multiplication-then-division taken as one rounded scaling,! Arithmetic overflowandMissing ) inserted for expression-- with register operands read when the program runs, in an assignment, a conditional and a\messagealike.\glueexprand\muexprthe same way over glue, and\gluestretch,\glueshrink,\gluestretchorder,\glueshrinkorder,\gluetomuand\mutoglue.- e-TeX's and LuaTeX's version numbers and strings (
\eTeXversion,\eTeXrevision,\luatexversion,\luatexrevision, ...),\currentgroupleveland\currentgrouptype,\showtokens,\showgroups, and the four interaction modes with\interactionmode: global, ending the terminal line, and batch mode withholding what the lowerer would print. - e-TeX's and LuaTeX's parameters -- every integer one luatex has
(
\tracingassigns,\lastlinefit,\suppresslongerror,\outputmode, ...), the page dimensions and offsets,\mathsurroundskipand\everyeof-- assigned, read and scoped like TeX82's, from luatex's INITEX values.\suppresslongerrorlets a\parinto a macro argument, and\hoffsetand\pageleftoffsetset each other an inch apart. What the stomach and the backend make of the rest is theirs. - LuaTeX's conditionals
\ifabsnum,\ifabsdim,\ifincsname,\ifprimitiveand\ifcondition, decided in running text, in an\edefand in a\message, and over a register the run holds;\begincsnameand\lastnamedcswherever they expand;\immediateassignmentand\immediateassignedin running text. \tracinggroupsand\tracingassignswrite what luatex writes, in its order: a group is reported entering and leaving (after its restores, as luatex'sunsavedoes), and an assignment aschanging,intoorreassigning-- a macro, a\let, a\chardef(which is\relaxwhile its number is read), a parameter, a token list, and a count or dimension whose value the lowerer follows. Tracing output after the last message reaches the terminal.- LuaTeX's category-code tables:
\catcodetable,\savecatcodetableand\initcatcodetable, with luatex's checks (! Invalid \catcode table) and a group restoring a\catcodeinto the table it was made in. Every prefixable LuaTeX command takes\global. \endreports the conditionals it found incomplete, with the line each began on (§1335), and run as a LuaTeX (-progname=luatex, or any LaTeX document) ends as luatex does: the file's)against the last message, and the open groups listed.\currentiflevel,\currentiftypeand\currentifbranch, for a conditional the lowerer decided and for the arms of one the run decides alike.\scantokensand\scantextokens: the text read again as a file, a line at a time under the category codes in force, with\everyeofat its end.- The engine's call points for LuaTeX's input callbacks (
src/input_hooks.rs):find_read_fileandopen_read_filewhere\inputand\openinopen a file, andprocess_input_bufferon each line of the document and the files it inputs as the mouth reaches it, without its line end. - Groups, which scope the macro table AND the registers they write; a
register assigned
\globalinside one keeps what the last global assignment left, even when a local one follows it (§283). - Registers:
\count,\dimen,\skip,\toksand\muskip, with`xcharacter codes and\advance/\multiply/\divide.\dimen0=10pt \advance\dimen0 by 5ptreads back15.0pt;\skip0=3pt plus 1filreads back with its stretch. The dimension, glue and token parameters oftex.web§247, §224 and §230 --\hsize,\parindent,\baselineskip,\thinmuskip,\everyparand the rest -- are registers in the same sense, at INITEX's zero: assigned, read, coerced and restored by a group like\dimen,\skipand\toks. On the string path they are stores: the breaker's measure isLayout's, not\hsize, and\everyparis not inserted when a paragraph starts. A plain document's list path (--dvi) reads them as tex does,\everyparin front of the token that started the paragraph (§1090-§1091). - Box registers and the lists they hold, built while the program runs
(
src/stomach.rs):\setbox,\box,\copy,\lastbox,\vsplit,\hbox/\vbox/\vtopwithtoandspread,\raiseand its siblings,\unhboxand its siblings, glue, kerns, penalties, rules, leaders, discretionaries, marks,\insert,\vadjustand\shipout, with\font\x=cmr10 at 12ptloading the.tfmand the main loop of §1034-§1040 applying its ligatures and kerns. A\setboxin a group is undone at its end and a\globalone is not (§283);\wd,\ht,\dp,\badnessand\lastpenaltyread as registers wherever a number or dimension is read,\wd0=changes the box in place,\ifvoid/\ifhbox/\ifvboxare run-time branches, and\showboxprints §198's display.tests/casesholds three documents of them against tex's terminal output. In a plain document only: a LaTeX prelude defines\hboxas a text macro. - The integer parameters of
tex.web§236 —\tolerance,\escapechar,\mag,\timeand the rest — at INITEX's values (§240) and the local clock's date (§241): assigned, read by\the/\numberand wherever a number is scanned, changed by\advance/\multiply/\divide, scoped by groups.\escapechardecides what\string,\meaningand\messageprint in front of a name, and-1prints nothing.\endlinecharis what the mouth appends to each line as it reads it (-1appends nothing, and an assignment changes the NEXT line), and\newlinecharis the character a\messageprints as a line end; a control character prints in^^notation. The--pdfline breaker reads\tolerance,\pretolerance,\linepenalty, the hyphen penalties and the demerit weights a document set, paragraph by paragraph and under--dvitoo, and its page breaker reads\clubpenalty,\widowpenaltyand\brokenpenalty. \errmessage, reported as tex reports an error and carried on from.\message, and\immediate\openout,\immediate\writeand\immediate\closeouton LuaTeX's 128 write streams. A file gets LuaTeX's name --.texadded when the name has none, a braced name keeping its spaces,\jobnameexpanded inside it -- and LuaTeX's bytes, every character raw and\newlinecharending the line;tests/write_streams.rsholds the files byte for byte against luatex's. A closed stream or one past 127 writes to the terminal and a negative one to the log alone. A file written in the run is read back by a later\inputor\openinas it now stands, not as the run before left it. On a plain document's list path (--dvi) a\write,\openoutor\closeoutWITHOUT\immediateis a whatsit node of its page and is carried out when that page ships (§1373), in the order the page is drawn -- a\writeto the terminal between the page's[1and its], reading its registers then -- and a\specialis a whatsit node whosexxxthe DVI carries where it stands. On the string path all four wait for a shipout that never comes, and [0x06] says why.\input, which is what every real document does first: the file is read where it is named, and its own(./name.tex …)nests inside the outer one's.\endinputends it with the line it is on.\openin,\closein,\readand\ifeof: a stream opens on the file the name finds, as\inputfinds one, and\readtakes it a line at a time (more while a brace is open), closing it when a read finds nothing left. A\readfrom a closed stream reads the terminal: the CLI has none, so it stops the run, and an embedded run reads the lines of its input (see Embedding).- Verbatim environments —
verbatim,Verbatimand the fancyvrb family,lstlisting,minted,alltt,filecontents— where the catcodes are suspended, so#,&and\inside one are characters rather than markup. - The LaTeX layer of [0x06]:
\newcommandand its three relatives dispatched natively,latex.ltx's counters, cross references, options, footnotes, captions and bibliography ported intosrc/latex/kernel.tex, a real load attempted for every\usepackage,\makeatletteras the catcode change it is. - Maths.
$…$,$$…$$,\(,\[,equationand its relatives are parsed into an mlist and converted bymlist_to_hlist: styles, sub- and superscripts with §756-§759's shifts,\over/\frac,\sqrt,\left…\rightwith delimiter growth,\limits, the operator names, and the Greek and symbol tables plain.tex names — with §764's spacing table ported verbatim. - Lua.
\directluaruns its chunk in PUC-Lua 5.3, withtex,token,texio,statusandluatexbasereaching the engine's real state. The state is built the way LuaTeX'sluainterpreter()builds it: LPeg, slnunicode (unicode), LuaFileSystem andbit32compiled from the C sources LuaTeX itself ships (vendor/luatex/),zlibandgzipover zlib,md5andsha2,fioandsio, thestringandosextensions, the compatibility functions TeX Live's Lua is built with (module,package.seeall,math.pow,__ipairs),kpse,lua(bytecode registers,\luabytecode),callback(LuaTeX's registry and slot numbers) andstatusanswering for the LuaTeX 1.24 interface.requirefinds a module as a file of kpathsea'sluaformat, throughkpsewhich, so lualibs loads as it does in luatex. A library function a package keeps in a local reaches the input of whichever chunk later calls it.nodeis LuaTeX's whole vocabulary on both faces (userdata andnode.direct), with attribute lists and the properties table;texreads and assigns the internal parameters texrs holds and the catcode, lccode, uccode, sfcode, mathcode and delcode tables through the engine's own assignments;fontdefines fonts from Lua tables and reads TFMs as luatex does;pdfhands its dictionary strings,\pdfvariables and objects to the backend that\pdfextensionwrites through, so they reach the file. A list callback (hpack_filter,pre_linebreak_filterand the rest) registered from a chunk is called by the stomach while the program runs, with luatex's arguments, and its answer is what gets packed. TeX Live's luaotfload loads under ltluatex and registers its definer ondefine_font, as in luatex. --dvi: a page. A plain document is set astexsets it, plain.tex preloaded and every list built by the stomach, and its DVI is tex's byte for byte on every document dvi-parity measures and on the formulas and displays oftests/math_cases. A LaTeX document is measured in a real font (.tfm), broken into total-fit lines by the same breaker as--pdf, packed into boxes and shipped throughhlist_out/vlist_out-- a fixed number of lines to a page, and no maths (a formula's scripts are dropped;--pdfsets them).\label,\refand\pageref, resolved against the pass that finds the pages —\refgives\@currentlabelas the class's\the<counter>writes it: a heading's number (1in an article,1.1in a report or book, lettered after\appendix), a float caption's, or the equation number a numbered display sets beside it (equation, thealignfamily per row,\tag,leqno); a float or display is a group, so its number does not outlive it.\counterwithin/\counterwithoutare the kernel's;\numberwithinand\eqrefexist only behind\usepackage{amsmath}, as under pdflatex.\pagerefgives the page it fell on. Worth knowing what this is NOT evidence of: the corpus has 88,341\label{and zero\ref{or\pageref{, because Pandoc writes labels and never references them. So this moves no book in the sweep. It is correctness for LaTeX documents generally, not a corpus result, and re-running the sweep to look for a change will find none.- OpenType shaping as LuaHBTeX's
mode=harfdoes it (src/shape.rs), ported from luaotfload'sluaotfload-harf-define.luaandluaotfload-harf-plug.luawith HarfBuzz through harfrust: a font's characters table and parameters at a size, and a list cut into runs, shaped, and turned back into glyph, kern and glue nodes -- discretionaries a ligature spans reshaped as luaotfload reshapes them,/ActualTextspans where a cluster is not one glyph. Checked list by list against luahbtex (tests/harf_parity.rs, frozen byscripts/harf-freeze.sh): Latin in a CFF and in two TrueType faces, Greek, Cyrillic, Arabic right to left, Devanagari and Han agree on every glyph's slot, offsets and dimensions and on every kern, glue and discretionary. A\fontrequest string is read as luaotfload reads it (src/fontreq.rs, thefont_requestgrammar ofluaotfload-parsers.lua):file:,name:,[path], a subfont, the slash modifiers and the feature list. - OpenType fonts as luaotfload's default
mode=nodeloads and processes them (src/otf/), ported from the ConTeXt fontloader luaotfload carries rather than from HarfBuzz: the font tables read asfont-otr,font-cffandfont-dspread them and unified asfont-oupunifies them, so a font's characters table -- slots, private slots, widths, heights, depths,/ToUnicode-- and its parameters come out as luaotfload hands them to the engine (tests/node_chars.rs); and a list processed asfont-otsandfont-otjprocess it: the features a request asks for, worked out asapply_default_featuresworks them out, applied lookup by lookup with their discretionary handling, the positioning injected as font kerns, offsets and wider spaces, and luaotfload's own processors around them (NFC normalisation, the Arabic joining analyzer and mark reordering,notdef, default ignorables, variation sequences,tligand the other extra features). Checked list by list against luahbtex inmode=node(tests/node_parity.rs, frozen byscripts/node-freeze.sh): Latin with kerns, ligatures, space kerns and hyphenation discretionaries in CFF and TrueType faces, optional features and alternates, Greek, Cyrillic, combining marks, Han and ideographic variation sequences, and Arabic right to left. A variable TrueType font is loaded at the instanceaxis=orinstance=asks for -- advances, bounding boxes from the varied outlines, and the GPOS values that vary -- and checked against luahbtex at several instances (tests/node_var.rs). A math font loaded as unicode-math loads one (script=math) carries luaotfload's MathConstants, variant chains, extensible recipes, italics, top accents and math kerns, checked against luahbtex for Latin Modern, STIX Two, TeX Gyre Termes and Pagella Math (tests/node_math.rs).name:and unprefixed requests resolve through luaotfload's own names database and its lookup cache, read where luaotfload keeps them (src/otf/names.rs, checked request by request against luaotfload's resolver bytests/node_names.rs). Neither this nor the harf shaper is on the typesetting path yet; [0x06] says what is missing. - The font programs luatex embeds for an OpenType font, as libraries for the
PDF writer: a TrueType face's
/FontFile2subset (src/otf/ttsubset.rs, writetype2.c'smake_tt_subsetwith sfnt.c's table writer), a CFF face's/FontFile3subset (src/otf/cffsubset.rs, writecff.c'swrite_cffandwrite_cid_cff: name-keyed fonts made CID-keyed, subroutines inlined), each with its/CIDSet, and the CID font's/ToUnicodeCMap (src/otf/cidcmap.rs, tounicode.c'swrite_cid_tounicode). Each is compared byte for byte with what luahbtex writes for the same glyphs (tests/ttsubset.rsandtests/cffsubset.rs, frozen byscripts/ttsubset-freeze.shandscripts/cffsubset-freeze.sh): DejaVu Sans, Arimo and Noto Serif; Latin Modern Roman, TeX Gyre Termes, STIX Two Math and the CID-keyed Harano Aji Mincho. --pdf: the PDF itself, and the better half of the typesetter. Lines are chosen by minimising total demerits across the whole paragraph (tex.web§813-§890) with Liang hyphenation, and set to the measure with PDF'sTw, which is what makes pricing glue usable at all. The file is written as LuaTeX's backend writes one: its objects numbered, ordered and packed into object streams as luatex does, its streams deflated by zlib at luatex's level, a TeX font's widths and descriptor taken from its TFM, its subset tag and the file's/IDmade by luatex's recipes. LuaTeX's page-stream writer is ported too (src/pdfpage.rs: text inBT/TJwith kerns, rules,pdf:specials, save, restore and matrices), and so is its shipout (src/pdfship.rs:hlist_outandvlist_outover the stomach's box trees). Plain luatex's page forHello world., built as its output routine builds it and shipped there, is luatex's file object for object but for the font program and the information dictionary. No run ships through it yet: a--pdfpage is still drawn from broken strings.- LuaTeX's backend commands.
\pdfvariableexpands to the parameter its key names, assigned, read and grouped like any;\pdffeedbackanswerslastobj,lastannot,lastlink,pageref,colorstackinit,versionand the rest;\pdfextensionreads every one of luatex's keywords. Objects, outlines (with the destinations they name, in a name tree),info,catalogwithopenaction,names,trailer,pageattr,pagesattrandpageresourcesreach the file numbered, ordered and spelled as luatex writes them: measured on a document using all of them, every object agrees with luatex's byte for byte but the content stream, the font program and the information dictionary (tests/pdf_backend.rs). - Readers for the binary formats a TeX installation is made of, each printing
what it read:
.tfm,.vf,.pk,.otf,.pfb,.map,.enc,.dvi,.bib/.aux/.bst, and tar bundles.
[0x04] Intercepts
\intercept % => [in]HELLO-WORLD
\intercept % every sectioning macro, including
% the ones a package defines later
\intercept % => <<HELLO-WORLD>>
Advice on macro expansion — before, after, around, with \proceed
standing for the original expansion inside an around handler. The pattern is a
glob over macro names, which is what makes it useful on a macro package: the
advice is registered before the macros it will catch exist.
Expansion is a compile-time act here, so advice is woven into the token stream and is undone by the group that registered it, like any other assignment. A handler that calls the macro it advises does not weave itself — a call inside advice is not advised.
[0x05] Inline Rust
\rust
\count1=21
\message % => 42
The block is compiled by rustc, loaded, and its exported functions become
callable. Its body is Rust, not TeX, so it is lifted out of the file before
the mouth reads it — #, {, } and & are category codes the mouth would
act on. A call is a number wherever TeX reads one: a register assignment, an
arithmetic operand, a conditional, or a \message body.
The compiled library is cached by body hash, so only the first run pays for the compile, and a block that does not compile stops the run with rustc's own diagnostic rather than a missing-function error later.
[0x06] What does not
There is a stomach now. Both outputs break a paragraph as tex.web §813-§890
does: every feasible set of breakpoints is priced by how far each line's glue is
from its natural width, the cheapest set wins, and Liang hyphenation widens the
places a line may end when no set between words is good enough. --dvi took the
first break that fits until it could ship a box tree, because a breaker that
decides some lines should be SHRUNK had nowhere to put that answer;
src/shipout.rs ports §619-§640's hlist_out and vlist_out, so the glue goes
into the file at the width hpack set it to (§625) and the answer has somewhere
to go. The .tfm's ligature and kern program is on that path too (§906-§911's
reconstitute), so tex's fi ligature is texrs's, and its quotation marks
and dashes with it — every document the two engines both set now agrees with
tex on its text and its structure.
--pdf breaks its pages by penalty too, over the whole document rather than
page by page: \widowpenalty and \clubpenalty keep one line of a paragraph
off the top and the bottom of a page, \brokenpenalty discourages ending a
page on a hyphenated line, and a heading is never left at the foot of a page
away from the text it introduces (\@secpenalty, and the \nobreak
\@startsection writes after a title).
Under both, tex.web's own machinery is ported rather than approximated:
hpack/vpack with §108's integer badness and §658's order-of-infinity glue
setting (src/pack.rs); the boxes a document can nest — \hbox, \vbox,
\vtop, \raise, \unhbox, \leaders, \lastbox, \unskip
(src/box_.rs); lines assembled from breakpoints with \rightskip,
\parshape and \vadjust (src/postline.rs, §877-§890); and a page builder
with \insert, \vsplit, \topmark/\firstmark/\botmark and an output
routine over \box255 (src/page.rs, §967-§1028). Maths is there too: $…$,
$$…$$, \(, \[ and the display environments are read into an mlist and
converted by mlist_to_hlist (§719-§767), out of cmr/cmmi/cmsy/cmex's
fontdimens (src/math.rs).
A plain document's --dvi, and its message run, now take the list path.
plain.tex is preloaded as tex preloads it, and the stomach builds every list
(src/stomach.rs, [0x03]). Paragraphs are broken from their node lists
(src/parbreak.rs, §813-§876, with §891-§918's hyphenation), plain's
\output runs when the page builder fires it, and the DVI is written as
§583-§643 write it (src/dvi/out.rs). Every document
cargo run --bin dvi-parity measures comes out byte for byte as tex writes
it. Formulas are built there too: math mode's commands make an mlist
(§1136-§1193, src/stomach/math.rs), mlist_to_hlist sets it from the
families the document loaded, and a display is placed with its equation number
as §1199-§1205 place it. The documents in tests/math_cases come out as tex's
DVI byte for byte, and so do \accent (§1123-§1125), \parshape (§1248)
and alignments (§768-§812): tables, and plain.tex's displays and matrices
built with \halign (tests/align_cases). BUGS.md lists what that path
still lacks.
--pdf and every LaTeX document still take the string path. There the page
breaker reads its penalties once per document, and the shipper is handed line
boxes built from already-broken strings.
The same missing node list is why, on the string path, a \write,
\openout, \closeout or \special without \immediate does nothing yet, nor a \pdfextension literal, dest, annot, link, thread, save, restore, setmatrix,
colorstack or refobj, a \useimageresource or a \savepos. Each is read and built as the
whatsit §1341 describes, and Lowerer::ship_whatsit is §1373's out_what for
it -- the \write expanded when it is shipped, not when it was read -- but a
whatsit has to sit in the list of the page it is shipped with, and until pages
are built from node lists the whatsits wait in deferred_whatsits for a
shipout that never comes. \immediate does all three now. On a plain
document's list path the four have their nodes: a \special reaches the DVI
with its page, and the other three are done when their page ships.
On the DVI ladder nine of the ten documents reach STRUCTURE: the text and its
structure agree with tex, the bytes do not — see BUGS.md.
Some LaTeX, and Lua. texrs carries the part of LaTeX that lives in the mouth
and the expander, as TeX rather than as Rust, in two files compiled into the
binary: src/latex/prelude.tex is a stand-in — a macro that would have drawn
something yields its text instead — and src/latex/kernel.tex is a port of
latex.ltx, with the place each definition came from written above it and every
substitution named. Counters, cross references, \newenvironment, the option
machinery, footnotes, captions and the bibliography are ported. \documentclass
and \usepackage are no longer silently consumed: the file is found with
kpsewhich and its load is attempted, and a package that will not go through is
reported by name with the control sequence that stopped it — texrs: package xcolor needs \cs_new_protected:Npn — and a package's own \RequirePackages
are followed, so keyval is read before the graphicx that asks for it. Which
of the classes and packages real documents use load, and whether the page they
set says what lualatex's says, is a table scripts/package-status.sh measures
and writes into BUGS.md ("LaTeX, loaded the way lualatex loads it").
The genuine kernel is the way out of the stand-in, and texrs -ini -progname=lualatex latex.ltx is how far it has got: TeX Live 2026's own
latex.ltx, found with kpsewhich, read through its e-TeX and LuaTeX
checks, the texsys probe, the allocator and every register it \countdefs
past 255, LuaTeX's catcode tables and the UnicodeData.txt pass that gives
every letter its category, ltluatex.lua, and into expl3.ltx, which stops
on Lua the engine has not got yet (BUGS.md has the list, in the order the
kernel reaches it). What made that possible is that the frontend now follows
every register's value as the program assigns it (src/known.rs), and the
expander reads that table: a conditional over registers is decided where the
mouth reaches it, as §498 decides it, an \edef freezes \the\count,
\the\dimen and \the\skip, and a register only the run knows (a
\rustcall's) is the one case still decided at run time. Run as a LuaTeX —
-progname=luatex, lualatex or luahbtex, or any LaTeX document — texrs
has LuaTeX's 65536 registers of each kind, its attribute registers, and a
category code for every Unicode character; plain -ini keeps TeX82's 256.
10\p@ reads as 10.0pt, whether the factor is written out or arrives from a
macro (\def\x{10}\dimen0=\x\p@), in plain TeX and in a LaTeX preamble, as
luatex and lualatex read it: §453's <factor><internal unit> is implemented,
and so are em and ex, which are the font's \fontdimen6 and \fontdimen5
out of cmr10.tfm and agree with tex to the scaled point —
[em=10.00002pt][ex=4.30554pt] from both engines. A package that loads and then breaks what the
preamble already promised is reported rather than committed — measured, letting
calc through took the corpus sweep from 229 documents to 145. For everything
still refused, the report is the list of what each one wants. What is kept
out of them is the page: a type size in the class options sets the text at that
size on the leading LaTeX pairs with it, and [margin=...]{geometry} sets the
margins and the measure and text height they leave on the paper. \makeatletter
is a catcode change and works as one.
\directlua runs its chunk, in PUC-Lua 5.3 — the version LuaTeX embeds — and
what the chunk prints is read back as input, so \count10=20 a\directlua{tex.print(tex.count[10]+5)}b typesets a25b. The tex table
reaches the real registers and parameters, token reaches the input the chunk
stands in front of, and a chunk that fails stops the run with its Lua error as a
TeX error. The node lists TeX builds are the stomach's and exist while the
program runs, after the chunks have run while it was lowered: so a chunk cannot
reach tex.box[n], tex.getlist or the current list (node.write,
node.last_node), and those refuse by name; a callback registered from a chunk
is how Lua reaches the stomach's lists (BUGS.md lists which callbacks are
called). node.ligaturing, node.kerning,
node.hyphenating and node.mlist_to_hlist are not ported. Of LuaTeX's other
libraries, lang, vf, img, mplib, ffi and socket are not there, and a font a define_font callback
answers sets type from its characters' dimensions but does not yet reach the
PDF as the file it names. The shell is disabled (os.execute, io.popen and
kpse.check_permission answer as luatex -no-shell-escape does), and a file a chunk opens, removes or makes goes through kpathsea's openin_any/openout_any checks as in luatex, and an error
a Rust library function raises is a userdata that prints, concatenates and
matches like the string luatex's C would raise, but is not type string. See
BUGS.md for each.
The number, run against 274 .tex files — a real LaTeX/LuaLaTeX corpus of
Pandoc-generated books of 16,000 lines and up, with fontspec, TikZ,
\directlua and --include-in-header fragments, plus texrs's own fixtures
including the ones written to be refused: 229 of 274 run to completion, and
say 75,627,678 bytes of text.
Which 274: 167 from the publications tree scripts/publications.sh defaults to,
plus the .tex files then under tests/cases, examples/, tests/pdf_cases
and tests/dvi_cases. tests/cases has grown since, so the set is not
reproducible from HEAD alone. Point the sweep at a parent of both trees to
sweep the whole set; point it at the default and you are measuring the
publications tree alone, which is a different number and a different claim.
That number went DOWN when Lua started running: the same corpus was 266 of 274
before it. 42 of the 46 failures are one family of header fragments carrying an
unsubstituted @FALLBACK_LIST@ placeholder inside a \directlua, which is not
valid Lua — texrs used to "complete" them only because it consumed the chunk
unread. lualatex refuses those same files too (! LaTeX Error: \usepackage before \documentclass), so the drop is the engine reading what it used to skip
rather than a capability lost. Of the remaining four, three are texrs's own DVI
fixtures, which stop on \special, \font and \vfill, and one is written to
fail.
Two things decide that count besides the engine, and a re-run that ignores them
reports failures texrs did not cause. The sweep defaults to target/debug/texrs
and to a 120-second limit per document; a debug build is several times slower
than a release one, and 120s is short for a 16,000-line book. A run of mine over
the 167-file publications tree put 35 documents over that limit — and the first
of them, given 600s, finished and produced 780,253 bytes of text. Those are
rc=124, the harness killing the run, and the script now says so and prints the
re-run that settles it rather than listing them beside real refusals.
That is a measurement, so it is re-measurable rather than remembered:
Every document that fails is printed with the line it stopped on; there is no
allowlist, so a corpus carrying a file written to be refused shows it among
them. The sweep is where the LaTeX layer's faults have actually been found —
tests pin behaviour a sentence at a time and tests/cases pins byte-for-byte
parity with tex, and neither of them says whether a 16,000-line book
compiles.
"Run" is the exact claim: the mouth and the expander read the whole document and
produce what its text says. --dvi sets a page, --pdf writes the PDF itself,
and three of the things a document controls survive the trip:
-
Fonts.
\setmainfont{Arimo}is honoured, both ways a document writes it. A family the machine has installed is resolved throughfc-match; a document that ships its own font names a FILE instead, with fontspec'sPath=,UprightFont=andExtension=, and that is read too — in either spelling.\setmainfont{Arimo}[Path=…, Extension=.ttf, UprightFont=Arimo-VF]embedsCRYLIS+Arimo-VF, and so does\setmainfont{Arimo-VF.ttf}[Path=…]: a mandatory argument ending in.ttf,.otf,.ttcor.otc— in any case — names the FILE, relative toPath=, which is what lualatex does with it. An explicitUprightFont=/Extension=still wins where the document writes both.Scale=is applied, includingScale=MatchLowercase, which is the form a document actually writes: the factor is the ratio of the main face's x-height to this family's, read from OS/2'ssxHeight. Measured, with the same file differing only by the option:Scale=0.5 10 pt -> 5 pt \setsansfont{Orbitron} {\sffamily\Huge} = 24.787 pt the same with MatchLowercase 22.578 pt, a factor of 0.91090.9109 is Arimo's
sxHeightover Orbitron's, 1082/2048 over 580/1000 — the ratio the code computes, not the 0.9076 that lualatex's own rounded output divides to. A face whosesxHeightis absent — the versions of OS/2 predating the field — is left unscaled rather than scaled by a guess. Bold and italic are cuts of the main family and take its scale.pdffontsis how you tell: a document that meant to ship its own face and reportsHelvetica Type 1 nogot the fallback. The difference matters becausefc-matchALWAYS answers — asked for a font nobody has installed it returns its default, so a book whose fonts travel with it was set in whatever that default happened to be.Path=is written when the document is built and is regularly a directory that no longer exists, so a stale one is retried against the directory the document was read from, where the fonts are. The file is carried in the PDF as/FontFile2—pdffontsreportsKAZDEO+Arimo-VF TrueType yes yes, embedded AND subset — once, and referred to from every page; lines are broken on that font's own advance widths, out of itshmtxthrough itscmap. No kerning and no OpenType ligature reaches this path yet: the harf shaper and the node-mode processing of [0x03] are libraries beside it (BUGS.md, "OpenType fonts are not loaded as luaotfload loads them, yet"). A family nothing can be found for falls back to whichever of the fourteen carries the same metrics, not to Computer Modern. The face is SUBSET: only the glyphs the document actually set are kept, withcmap,glyf,loca,hmtx,maxpandheadrebuilt rather than copied, and each subset carries its own six-letter tag. A 5.6MB book comes out with five faces —Arimo-VF,Arimo-Italic-VF,Orbitron-VF,ShareTechMono-Regularand a systemArialUnicodeborrowed for glyphs the others lack — and the five embedded programs come to 183,944 bytes where the four bundled faces alone are 1,121,312 on disk. That pair is the claim about subsetting. The whole-file totals are a DIFFERENT claim and worth not confusing with it: this book is 2.6MB against lualatex's 968,737, but the file also carries a contents, folios, pictures, and five faces where lualatex embeds one subsetted Latin Modern. Read as a subsetting result it says the subsetter is poor; it is comparing two font sets. The descriptor states what the METRICS file states rather than the bounding box's extremes —/Ascent 694 /CapHeight 683 /Descent -194 /XHeight 431forcmr10, out ofcmr10.afm, which is what luatex writes for it byte for byte and none of which is anywhere in the.pfb. A Type 1 program still goes in whole. -
Colour.
\definecolor,\providecolorand\colorletbuild the palette and\color,\textcolorand\pagecoloruse it, in theHTML,rgb,RGB,grayandcmykmodels.\coloris a switch and ends with the group holding it;\textcolorputs the previous colour back. Under--pdfthis is PDF's ownrgoperator, and\pagecoloris painted under the text; under--dviit is thecolor push rgb R G B/color pop\specialpair that dvipdfmx and dvips both read. -
TikZ. A
tikzpicture(orpgfpicture) is drawn on the page under--pdf. Its body is read RAW — a picture is TikZ, not TeX, and the prelude used to consume\draw … ;and emit nothing, so every diagram in every document reached the page as blank space — and it travels the text stream as one marker the typesetter parses and draws. The picture takes its bounding box's height, computed the way PGF computes one (a curve's control points count, a stroked path grows by half its line width), so the paragraphs either side flow around it; a bare coordinate is a centimetre, which is PGF's own unit vector. Built against PGF's source rather than an approximation of it:--,-|,|-,..controls..,towithout=/in=/bend,rectangle,circle,ellipse,arc,grid,parabolaandcycle, painted by\draw,\fill,\filldraw,\path,\clip,\shade,\shadedrawor nothing, under either fill rule. Colour goes through the document's own palette;line width=, the named widths, the dash patterns, caps, joins andopacity=all reach an operator, transparency through a registered/ExtGState.\shade's axis, radial and ball ramps are painted byshthrough the path as a clip, with the/Shadingentry carried on the page.->,<-,<->,-stealthand-latexare drawn as the paths PGF draws them as, on a line shortened to make room;snake,zigzag,sawandbracedecorations replace the segment they are put on. Nodes carry text through the same typesetter the rest of the page uses, placed by any of the nine anchors or by the border at any angle, in a rectangle, circle, ellipse or diamond. Coordinates may be polar, named, a node's anchor, relative,pgfmatharithmetic orcalc;\foreachand nestedscopes are read. Not there: patterns (apattern=turns the fill off rather than hatching), the matrix and graph libraries,pics, and INLINE placement — lualatex sets a picture in the line where it stands and this gives it lines of its own.--dviand--textdraw no picture at all: DVI would need a\specialevery driver reads differently, and a picture has no words. -
The face axes compose now, and one fallback is left. Family, series and shape were a single slot until v0.6.0, so the last switch won and the earlier one was lost. They are three axes, as NFSS has them:
{\sffamily\Huge X} Orbitron-VF was Orbitron-VF {\sffamily\bfseries\Huge X} Orbitron-VF was Helvetica-Bold {\bfseries\Huge X} Helvetica-Bold unchangedThe middle row is the fix:
{\sffamily\bfseries\Huge}is the ordinary way to write a heading format, and the family used to be lost to the series. What the third row shows is what is left — asking for bold in a family that declares no bold cut gets base-14 Helvetica-Bold rather than a bold of that family. -
Ligatures and type sizes were here and are not. Both were listed as limitations in this section until v0.6.0.
\section,{\huge …}and{\Large …}in one file now emit three distinctTfsizes rather than one, and/'' ``/--/`---` now set “ ” – — where they used to set the marks doubled. Extracting the same source through both engines gives the same line:lualatex en – dash, em — dash, open “ close ”, single ‘ and ’ texrs en – dash, em — dash, open “ close ”, single ‘ and ’The exclusions are right too:
\texttt{--flag}keeps both hyphens,-{}-stays two, and----is an em dash followed by a hyphen.Recorded rather than deleted because a claim was derived from the size limitation and does not survive it: this README argued that headings set at body size were why a book came out with fewer lines AND more lines per page than lualatex sets it. With per-run sizes that reasoning is void, and no page ratio was ever published here — see the note under the corpus figure about which numbers are facts about the engine and which are facts about a run.
-
Images.
\includegraphicsembeds the file and reserves the room it takes. Measured on the same two documents that used to prove the opposite — one with a figure, one without, a real PNG present:with a figure /Subtype /Image present, "After." baseline y = 523.2 without no image, "After." baseline y = 703.2180pt of room reserved, and the text below moves down for it. The baselines are the evidence rather than the file sizes: a byte count moves with every feature the writer gains, where the offset is the thing being claimed. An image given no size is bounded to the measure and the text height rather than set at the file's own — a diagram exported at 1600 pixels is 1600 big points wide, which is wider than the page. A file the reader cannot rasterise (a
.pdfnamed where a.pngsits beside it) falls back to a sibling of the same name.This entry said the opposite until v0.6.0, and said it with numbers. Three faults were stacked and the first hid the other two: the primitive was wired to nothing,
\pandocboundedboxed every figure without setting the box, and once figures did reach the page they were placed at the file's own size.
What --dvi still does not do is draw a picture, or set the maths of a LaTeX
document. A plain document is set as tex sets it, formulas included. A LaTeX
one takes the string path, which
stacks a fixed number of lines on each page where --pdf prices the whole
document. A draft reads correctly either way; a book being sold on its
typography should still be set by an engine that has been doing it for forty
years, and scripts/texrs-pdf says the same thing where a build would meet it.
That script is a .tex to a .pdf, not a pandoc --pdf-engine: pandoc
validates that flag against a fixed list of engines it knows and refuses any
other name, so texrs cannot be named there under any wrapper. It does not need
to be — a pandoc book build already writes the .tex on its first pass, and
that is the whole input an engine takes, so replacing the second pandoc call
also drops a second markdown-to-LaTeX pass over the same document.
[0x07] How it runs
texrs is a fusevm frontend, not an interpreter: mouth → expander → command
stream → fusevm bytecode → the VM runs it. A count register is a VM slot, so
\advance\count0 by 5 is GetSlot / LoadInt / Add / SetSlot — native ops the
JIT can compile. \ifnum is NumGt + JumpIfFalse, a real branch.
A conditional whose truth depends only on the macro table (\iftrue, \ifx)
is folded while lowering instead, because there is nothing for the VM to test.
tests/lowering.rs asserts the emitted bytecode rather than the printed output —
output parity alone would not distinguish a frontend from a tree-walker.
[0x08] Parity
There are three contracts, against two oracles, and only the first is a byte comparison that already holds.
The \message stream is compared byte-for-byte against the real tex binary.
No expectation is written by hand:
No expectation is written by hand in the other direction either: the freeze
that records them, cargo run --bin parity -- --freeze, re-records from tex
and not from texrs. A reader who assumed it captured texrs's own output would
have the whole gate backwards — it would then be asserting that texrs still
agrees with itself.
Both harnesses read the engine version they were measured against out of
BUGS.md and refuse to run against a different tex: a mismatched oracle does
not fail loudly, it reports a different set of divergences, which reads exactly
like a regression.
The corpus is small and deliberately awkward — ## inside a nested \def,
catcode changes mid-file, \csname built from a macro, control-word space
swallowing, conditionals nested inside a \message body, \ifcase,
\expandafter, \edef freezing a register, a \count assignment scoped by a
group. Every case is in parity except the ones tests/known_gaps.txt names,
and the gate fails both on an unlisted divergence and on a listed case that has
started passing, so the list cannot go stale.
LuaTeX's own primitives -- e-TeX's, LuaTeX's, the ones TeX82 has not got -- are
compared against luatex the same way, by the cases in tests/luatex_cases:
INITEX luatex with every extra primitive enabled under its own name, the state
a LaTeX format starts from, against texrs run as a LuaTeX (-progname=luatex).
Each case names the primitives it exercises, the gaps are listed in tests/luatex_known_gaps.txt under the same two-way rule,
and bash scripts/primitive-census.sh measures every primitive luatex has
against texrs and writes the verdicts into BUGS.md:
The page is not compared that way, because nothing reaches byte equality yet.
tests/dvi_parity.rs and tests/pdf_parity.rs are RATCHETS instead: each
document has a rung it last reached, recorded in tests/dvi_floor.txt and
tests/pdf_floor.txt, and the gate fails when one drops rather than asserting
agreement it does not have.
A rung is only half a ratchet: climbing one changes nothing until it is recorded, and the commands that do it are
which is the opposite direction from parity --freeze. That one re-reads the
ORACLE; these two re-read texrs, and what they write down is how close it got.
Running either after a change that lost ground would lower the floor and retire
the very regression the file exists to catch, so they are run deliberately after
a climb rather than as part of a build.
DVI NONE < PARSES < PAGES < TEXT < STRUCTURE < BYTES
PDF NONE < PRODUCED < PAGES < PAGESIZE < TEXT < LINES < FONTS < BYTES
Ten documents on each. Today the DVI floor stands at STRUCTURE for nine of
them, and the PDF floor at FONTS for seven and TEXT for two, with
empty_document.tex the only case at BYTES, and the only one at NONE on the
other axis. The oracle differs too: DVI is
against tex, PDF against luatex, and both skip loudly rather than passing
when their oracle or the tools that read the output are missing.
Byte equality is a reachable goal for DVI, which carries no fonts and no
compression, and a distant one for PDF: Hello world. is 224 bytes from tex
against 192 from texrs, where the same document in PDF is 11,729 from luatex
against 14,592. The file is laid out, numbered and compressed as luatex lays it
out (src/pdfdoc.rs), so what still differs is inside the objects: the page's
content stream, which waits on pages shipped from node lists, and the font
program; BUGS.md has the object-by-object count.
[0x09] Fuzzing
Hand-written cases only cover what someone thought to write down.
parity-fuzz generates seeded programs confined to the implemented subset and
runs each through both engines. Each program packs 40 independent probes,
because the oracle is the expensive part — a tex invocation costs ~0.5s of
process start and format load, so one construct per invocation would spend the
whole budget on startup. On a divergence the probe list is minimized to the one
that actually diverges before it is reported, and every program is a pure
function of its index, so a finding replays exactly with --seed N --once.
The probes cover register arithmetic on counts, dimensions, glue and token
registers, the \if family, \afterassignment, \aftergroup, \futurelet,
\expandafter, \csname, case shifting, \string/\meaning, catcode and
\escapechar changes inside groups, and \edef freezing. The oracle loads the
plain format and texrs starts from INITEX, so a probe may not depend on a
catcode or a macro only plain sets.
fuzz/ is a cargo-fuzz crate (targets lex, lower, run) looking for panics
rather than divergences. tests/fuzz_smoke.rs replays each target on its seed
corpus under stable Rust, and tests/fuzz_mass_replay.rs points the mouth at
generated mutations of every .tex in the tree, so cargo test still exercises
the harness on a machine with no nightly toolchain.
[0x0A] Benchmarks
cargo bench separates the stages, because one end-to-end number cannot tell a
slow expander from a slow VM, and sweeps document size, because a frontend
that is quadratic in the number of macro calls looks fine on a small file and
stops being usable on a real one. That sweep has already earned its keep: it
caught a line-number lookup that was O(n) per token, and fixing it took an
800-statement document from 248 ms to 11.7 ms.
bench/compare.sh is the comparison that says whether the engine is fast
rather than where its time goes, and prints the two caveats with the numbers:
tex loads the plain format on every run while texrs loads nothing, and the two
no longer do the same work — texrs typesets and writes a PDF by default where
tex writes DVI.
Measured, at v0.3.0
These numbers are from v0.3.0 and have not been re-taken. They were measured
when texrs implemented the mouth and the expander and shipped no page at all —
before the line breaker, the page builder, fonts, images, maths, TikZ and the
PDF writer. Every one of those does work per run that was not being done when
these were recorded, so the ratios below describe an engine that no longer
exists. They are kept, dated, rather than deleted or quietly refreshed: the
shape of the finding — where the time goes, and that a batch parallelises where
tex cannot — is what the section is for, and a number presented as current is
worse than one presented as historical.
Against tex 3.141592653 (TeX Live 2026) on a 10-core machine. Every figure
below is a median of repeated interleaved runs rather than a best-of, because
the machine was under other load while they were taken. Read them with the
caveats above, and with the one that has since changed: at the time texrs was
NOT typesetting and tex was.
One document (5.6 MB, 120k statements, 691k tokens):
| median | min | |
|---|---|---|
| texrs | 0.259 s | 0.147 s |
tex |
1.038 s | 0.791 s |
| 4.0x | 5.4x |
Where that time goes, which is why the mouth is not worth parallelising:
| stage | time | share |
|---|---|---|
| mouth (lexing) | 0.143 s | 22% |
| expander + lowerer | 0.308 s | 48% |
| code generation | 0.014 s | 2% |
| VM execution | 0.171 s | 27% |
Many documents (60 files, one thread each). This is the case that
parallelises, and the one tex cannot do at all: one process compiles one
file, so the comparison is against running it 60 times.
| median | |
|---|---|
| texrs, all cores | 0.198 s |
| texrs, one at a time | 1.827 s |
tex, one at a time |
54.076 s |
272x faster than tex |
Scaling with --jobs, on those same 60 documents:
| jobs | 1 | 2 | 4 | 8 | 10 | 16 |
|---|---|---|---|---|---|---|
| speedup | 1.00x | 1.85x | 2.71x | 4.96x | 6.73x | 5.58x |
Ten cores, so the fall at 16 is oversubscription rather than a limit in the
work. At v0.3.0 a single document was a single thread: this parallelised the
batch, not the engine. Line breaking, page drawing, and page encoding on the
--dvi path now fan out inside one document; see below and the 2026-10-17 measurement.
PDF assembly. pdf::document writes each font and picture once and
finds an already-written one by address before comparing contents, so a font
shared by every page costs one pointer comparison per use rather than a read of
the whole font program. Measured on a synthetic 600-page document with 40
embedded faces, that lookup went from about 1.9 s to about 0.13 s. A picture
repeated by value (each page holding its own copy) is matched first by size,
depth and data length, and the full comparisons that confirm those matches run
in parallel; two pictures that match that way and still differ send the
document back to the one-by-one comparison. Object numbers are handed out by a
serial walk in the order the old single loop added objects, and each page's
content stream and page dictionary are then built in parallel into the numbers
reserved for them; each page's content moves into its stream rather than
being copied there. The content streams are deflated at luatex's level 9 on
the pool before the file is laid out, each reading only its own page; the
/ID is luatex's, a digest of the time, the directory and the file's name, so
the file itself is no longer hashed. Measured 2026-10-17 on a 6000-paragraph
plain document (--pdf, release, best of 8 at load averages 25 to 30): 3.06 s
before the luatex layout and 3.11 s after, for a file of 176,967 bytes where it
was 1,918,708. Subsetting and the cross-reference table stay
sequential; output is byte-identical across runs and thread counts (set
SOURCE_DATE_EPOCH to pin the creation date). Medians of interleaved runs
at 8 threads, pdf::document alone: 55 to 16 ms on a 1401-page document
repeating twenty pictures 4000 times, 59 to 43 ms on a 3328-page mixed one, 14
to 13 ms on the 1001-page LaTeX document below, where the digest is most of
what is left.
Page breaking. plan_pages picks the least-cost sequence of page breaks
backwards from the end of the document, and only that choice is serial: where
a page started at a given line may end, and what each end costs before what
follows it is added, depend on that start alone. On more than one thread
those offers are gathered in parallel a block of starts at a time into
buffers kept from block to block, while the calling thread chooses from the
block gathered before it; on one thread each start's offers are gathered into one reused
buffer just before they are chosen from. The break penalties and the leading
of each line are read per line in parallel, and the walks that join them run
in order. The plan is the one the single walk made: the same page boundaries,
feet and repeated heads. On the 1001-page LaTeX document below, medians of
interleaved runs: the planner 21 to 11 ms at one thread and to 5 ms at 8, the
penalties 11 to 2 ms and the leadings 5 to 1 ms at 8.
Page drawing. Once the lines are broken
and cut into pages, drawing a page reads only its own lines, the colour, face
and size stacks as they stood where the page begins, and its folio. What a
run of lines does to a stack does not depend on what the stack holds: it pops
some number of entries it did not push (stopping at the floor every stack
keeps) and leaves the entries it did push, each of which is either named by its
marker or made from the entry in force (a face code modifies the face below, so
a chain of codes is a table over the twelve faces). So on more than one thread
each page's effect on the stacks is read in parallel, and a serial walk only
applies those effects in page order to record each page's start; one thread, or
a handful of pages, walks the lines in turn. The pages are then drawn
independently and collected in page order, so a single document's page content
scales across cores and the file is identical at any thread count. A document
of only a few pages is drawn on the calling thread. Each page also shares its
fonts by reference (Arc) instead of copying a font program per drawn run,
which was the larger share of this stage before the fan-out.
Line breaking. What one paragraph hands the next is only the colour, face
and size stacks, the centring flag and the list depth. On a pool of more than
one thread, each paragraph's effect on that state is read in parallel without
measuring anything (the stacks as for page drawing; the centring flag and the
depth are each set or left alone), a serial walk applies the effects in order
and records the state at the head of each chunk of paragraphs, and the chunks
fill in their own paragraphs' start states in parallel; the paragraphs are then
broken in parallel and collected in document order, and the paragraph space between
two is decided on the collected lines. One thread, or a document of fewer than
64 paragraphs, walks the paragraphs in turn. Liang's patterns and the
\hyphenation exceptions are held in a letter trie, as tex.web holds them,
so a word costs one table step per letter from each position rather than a
hash of every substring; it breaks every word of /usr/share/dict/words
where the hash table did. Measured 2026-10-17 on the 1001-page document below,
under load averages between 43 and 110, median wall went from 1191 ms to 839 ms
at 8 threads and from 1664 ms to 1462 ms at one; output is byte-identical to
the previous build at both. Reading the paragraph and page effects in parallel
then took the two prescans on that document from 28 and 23 ms to 7 and 2 ms at
8 threads (median of 11 interleaved runs, load averages 6 to 9; median wall
379 ms to 333 ms), and the page prescan from 22 ms to 10 ms at one, where it no
longer cuts every line into runs; PDF and DVI are byte-identical to the
previous build at 1 and 8 threads.
Lowering is serial, but a paragraph it has finished is finished, so on a pool
of more than one thread a PDF run of a source of 64 KiB or more breaks
paragraphs while the rest of the document is still being lowered
(typeset::Pipeline). The lowerer hands on the text of its document-level
commands as far as they can no longer change (an open text run up to its last
paragraph break); a job on the pool fills each formula slot as its batch of
settings comes back, and breaks every 64 or more finished paragraphs from the
state the ones before them left. The page builder then breaks only the rest.
Nothing is assumed to match: the paragraphs are used only when the text they
were read from opens the text the run produced (after \ref, \pageref and the
contents are resolved), and the families and the layout are the ones they were
measured with, the page penalties aside. A conditional or loop the run decides,
or a colour switch, stops the feed; a \setmainfont, a fallback family or a
geometry or class option read after paragraphs were broken sends the whole
text to the page builder, as before. Measured 2026-10-17 (median of 15
interleaved runs, load averages 9 to 15), median wall at 8 threads went from
345 to 313 ms on the 1001-page LaTeX document below, 887 to 837 ms on a
formula-heavy 2801-page one, 120 to 107 ms on a 394-page plain TeX one and 101
to 88 ms on a 200-page LaTeX one; at one thread nothing changes. PDF, DVI and
messages are byte-identical to the previous build at 1, 2 and 8 threads.
With the paragraphs broken alongside it, a large document's wall time follows
the lowerer's own speed, so the work it does per token was cut. The lowerer
looks at the last character of the text run rather than comparing suffixes,
offers its commands to the paragraph feed when they have grown (or every 1024
tokens) instead of after every token, asks the mouth for invalid characters
and the register tracker for owed marks only when there are some, and answers
a character that is not a math shift without calling into the formula reader.
A macro's definition is held by Rc, so a call no longer copies its parameter
text and body; the meaning table and the interned names hash with FxHash; the
mouth reads TEXRS_PARALLEL once per process; and a picture's nodes use one
shared copy of xcolor's palette instead of building it per node. Measured
2026-10-17 (medians of interleaved runs, load averages 8 to 13), Lowerer::lower
alone at 8 threads went from 148 to 101 ms on the 1001-page LaTeX document,
58 to 37 ms on the 394-page plain TeX one, 417 to 292 ms on a 3328-page mixed
one and 258 to 217 ms on an 800-page formula-heavy one (203 to 166, 48 to 32,
709 to 626 and 702 to 671 ms at one thread). Median wall at 8 threads went
from 372 to 323 ms, 826 to 729 ms and 569 to 564 ms on the LaTeX, mixed and
formula documents, and stayed at 130-134 ms on the plain one, whose wall the
lowerer does not decide; at one thread, 1043 to 1011 ms and 2518 to 2316 ms on
the LaTeX and mixed ones. PDF, DVI, stdout and stderr are byte-identical to the
previous build at 1 and 8 threads.
The formula reader was the larger part of what was left. A noad held a built
box inline, which made every field of every noad the size of a box; it is now
boxed, so a formula is a fraction of the bytes to read, copy and drop. A single
formula's spelling and setting read its one list in place instead of a
flattened copy, the spelling is built in one string, a symbol's math code is
looked up by the token's id in plain.tex's table hashed once rather than
searched per control sequence, the mouth reads a control word into a buffer it
keeps, and a group argument is read without its brace instead of having it
taken out after. Measured 2026-10-17 on a loaded machine, as the lowering
thread's CPU time (medians of interleaved runs, load averages 39 to 48 at 8
threads, 19 to 20 at one), Lowerer::lower went from 132 to 107 ms on the
1001-page LaTeX document, 235 to 129 ms on the 800-page formula-heavy one, 371
to 283 ms on the 3328-page mixed one and 45 to 42 ms on the 394-page plain TeX
one at 8 threads (186 to 151, 751 to 539, 696 to 527 and 37 to 34 ms at one
thread). Median wall (5 interleaved runs, load averages 21 to 23) went from
410 to 377 ms, 666 to 568 ms, 959 to 864 ms, 178 to 168 ms and 266 to 238 ms
at 8 threads on those four and the 200-page LaTeX one, and from 1110 to 1082,
2037 to 1832, 2689 to 2533, 473 to 426 and 349 to 347 ms at one thread. PDF,
DVI, --text output, stdout and stderr are byte-identical to the previous
build at 1 and 8 threads.
Pictures, images and formulas. Before the pages are fitted, every TikZ
picture is parsed again against the document's own face to size its nodes, and
every \includegraphics line has its lengths resolved against the measure and
the file it names. Each such line depends only on itself, the fonts and that
file, so once a document has eight or more of them the lines are remeasured in
parallel and collected in document order. The files are found and read before
that, once per distinct path (on the rayon pool when there are eight or more
image lines and more than one file), and every line naming a path shares the
one picture read from it, for measuring and for drawing; the PDF writer then
finds repeated uses to be one XObject by address instead of comparing their
pixels. Before, each use found and read its file twice and kept its own copy of
the pixels until the file was written. On a document of 4,000 figures drawn
from 10 photographs (release build, 18-core aarch64 macOS, load averages 20 to
27, medians of 9 interleaved PDF runs, 2026-10-17) wall went from 659 to 236 ms
at one thread and 304 to 159 ms at 8, and peak resident memory from 1.7 GB to
62 MB; the mixed document of text, pictures, images and formulas was unchanged
within noise (2324 to 2381 ms at one thread, 767 to 743 ms at 8). Measured
earlier the same day under heavy machine load, the remeasuring stage went from
about 150 ms to 19 ms with 6,000 pictures, 130 ms to
33 ms with 4,000 images, and 185 ms to 33 ms for a mix of both with formulas.
A picture's marker is also decoded through a lookup table rather than a search
of the base64 alphabet, which had been about a quarter of a drawing-heavy
document's serial time.
A TikZ picture's marker carries the height it reserves, measured by parsing the
whole picture against no font, and that parse was a third of lowering on the
mixed document below and two thirds on the drawing one. The height is still
needed (the contents and \pageref passes fit pages from it before pictures
are remeasured), but the parse reads only the picture's own options and body,
so the lowerer now writes the span with a slot and hands the source to the
batches formulas go in (see Formulas below): one index space, the same
32-item threshold, the same part-batch at the join and the same fill in
document order, including the page builder's early breaking. With fewer than
32 formulas and pictures together, or at one thread, everything is measured on
the lowering thread at the end. Measured 2026-10-17
(release build, 18-core aarch64 macOS, load averages 12 to 21, medians of 15
interleaved PDF runs; lowering-thread CPU from a separately instrumented build,
medians of 5):
| document | wall at 1 thread | wall at 8 | lowering-thread CPU at 8 | of it, pictures |
|---|---|---|---|---|
| mixed text, pictures, formulas | 2232 → 2226 ms | 724 → 634 ms | 230 → 159 ms | 81 → 4 ms |
| 6,000 pictures | 795 → 741 ms | 313 → 262 ms | 107 → 44 ms | 70 → 3 ms |
| LaTeX text and formulas, no pictures | 1027 → 1000 ms | 313 → 317 ms | 96 → 97 ms | none |
PDF, DVI, stdout and stderr are byte-identical to the previous build at 1 and 8 threads on every benchmark document.
Formulas. A formula is READ while the document is lowered, because reading it expands macros against the engine and consumes the input, but it is no longer SET there. The lowerer writes the formula's marker with a slot where the setting's spec goes and keeps the parsed mlist with what setting it depends on: the style, the math families at the type size in force, whether it is a display, its environment's columns and the measure for an equation number. What the lowerer reads back from its own output is the last character of the text run, for a ligature's first half or a face marker, and a marker ends the same way whatever its spec holds, so nothing it decides waits on a setting. After lowering the formulas are set, in parallel once there are 16 or more, and each spec is written into the slot its index names. Measured 2026-10-17 on a synthetic document of about 35,000 formulas (load averages 18 to 36, medians of 9 to 11 interleaved runs), wall went from 1128 ms to 760 ms at the default thread count and from 1331 ms to 1037 ms at 8, for 22% more user CPU (2450 to 3000 ms); at one thread it is unchanged (2352 against 2356 ms). Output is byte-identical to the previous build at 1 and 8 threads.
A formula's inputs are fixed once it is read, so its setting no longer waits
for lowering to end: formulas are gathered in batches of 32 and each full batch
goes to the rayon pool as it fills, while the lowerer reads on. At the end the
lowerer sets the last part-batch itself, collects the batches by the index of
their first formula, and fills the slots; the settings are freed on a worker.
A document of fewer than 32 formulas, or a run with TEXRS_THREADS=1, sets
every formula on the lowering thread at the end, as before. Setting reports no
diagnostics, so there is no message order for the overlap to change. Measured
2026-10-17 (load averages 16 to 33, medians of 9 interleaved runs, PDF): the
35,000-formula document went from 1089 ms to 820 ms at 8 threads and 978 ms to
778 ms at the default count; a mixed text, picture and formula document from
1510 ms to 1317 ms at 8 and 1304 ms to 1245 ms at the default. PDF and DVI are
byte-identical to the previous build at 1, 2 and 8 threads.
The workers' own cost was then the limit: setting a formula spent about half
its time copying and freeing. mlist_to_hlist copied every list it converted
(and, recursing, every sublist again), copied each noad and the one after it
to look at them, packed a copy of each noad's hlist only to read its height and
depth, and copied every hlist once more in the second pass. Now a list is
copied only when a \mathchoice in it is spliced, a noad only when a make_*
procedure rewrites one of its fields, heights and depths are measured off the
list in place, the second pass moves each hlist out, hpack keeps the list it
is given unless an adjustment list takes nodes out of it, and a spec's numbers
are spelled without core::fmt. Measured 2026-10-17 (load averages 14 to 37,
medians of 9 interleaved runs, PDF): setting the 35,000-formula document's
formulas went from 377 ms to 143 ms of CPU on one thread and from 720 ms to
295 ms summed over the workers at 8; the lowerer's wait for the last batches
at the join went from 22 ms to under 1 ms. Wall went from 1660 ms to 1402 ms at
one thread and 488 ms to 443 ms at 8; on the mixed document from 2307 ms to
2081 ms at one thread and unchanged at 8 (694 against 695 ms), and on a LaTeX
document with few formulas from 1011 ms to 950 ms at one thread and unchanged
at 8 (307 against 313 ms).
At 8 threads lowering itself and the page drawing after it are what the wall
waits on now, not the formulas. PDF, DVI, stdout and stderr are byte-identical
to the previous build at 1 and 8 threads.
DVI pages. On the --dvi path a page is its own lines and its number, so
each page's box is built and its bop..eop commands are encoded by a writer
of its own, in parallel; the pages are then appended in order, which is where
each bop's pointer to the previous page is filled in and the postamble's
maximum height, width and stack depth are taken over the pages. The preamble,
font definitions and postamble are written serially. The file is byte-identical
to the single-writer output at any thread count, and a document of only a few
pages is encoded on the calling thread.
Setup and writing out. Outside the lowerer, a LaTeX run's longest serial
stretch was its preamble: each package the document names is first tried in a
throwaway lowerer of its own (see src/latex/load.rs), one after another, and
the register and counter scans read the whole source once per command name.
The tries read nothing another writes and are memoised by name, so on more
than one thread they are made side by side, at most TEXRS_THREADS at once,
and the preamble is then assembled from them in the document's order as
before; the per-command scans of a source of 64 KiB or more run on the pool
and are joined in command order. The code generator's string pool hashes with
FxHash. Measured 2026-10-17 at 8 threads (phase medians of 9 runs, load
averages 13 to 17), the preamble went from 13.4 to 5.7 ms on the 200-page
LaTeX document and from 21.2 to 10.1 ms on the 1001-page one, code generation
from 2.0 to 0.9 and 10.1 to 5.5 ms, and the PDF's stream copy, digest and
object stream from 12.2 to 8.3 ms on the 1001-page one. Median wall (15
interleaved runs, load averages 9 to 13) went from 69.6 to 62.4 ms and 244.3
to 229.7 ms at 8 threads on those two, 82.3 to 79.9 ms on the 394-page plain
TeX one and 13.9 to 13.0 ms on a one-line document; at one thread, 963.7 to
914.9 ms on the 1001-page document and within a few milliseconds on the rest.
PDF, DVI, .aux, stdout and stderr are byte-identical to the previous build at
1 and 8 threads. What stays serial at 8 threads, largest first on the 1001-page
document: lowering (about 100 ms), the SHA-256 over the file (about 8 ms), the
page planner's choice, code generation, process start and exit, and the file
write.
Bibliographies. -X bibtex runs an ITERATE or REVERSE of 256 or more
entries in chunks at once. An entry's own variables belong to it, so all one
entry's run can hand the next is the global variables, the stack and the
half-written output line. Each chunk after the first starts from the globals as
they stood before the walk, with an empty stack and line, and records which
globals it read before writing them. A chunk is kept only when the chunk before
it ended with an empty stack and line and no earlier chunk wrote a global it
read first; the first chunk that fails, and every one after it, runs again in
order. A walk whose first entry reads a global and then writes it (a running
maximum such as longest.label, alpha's previous label) stays in order from the
start. Database and .aux reading stay serial: the .bib of 50,000 entries
below reads in about 70 ms, and an @string changes the meaning of every
record after it. Before the parallel walk the database reader, the .aux
reader and the entry loading found repeated keys, citations and
cross-reference targets by scanning every earlier entry, which was quadratic in
the entry count; they now use hash sets. Measured 2026-10-17 (release build,
18-core aarch64 macOS, load averages 18 to 36, medians of 5 to 7 interleaved
runs) on a generated .bib with plain.bst, wall / user CPU in ms:
| entries, cited | before | 1 thread | 4 | 8 | 18 |
|---|---|---|---|---|---|
| 50,000, 25,000 | 11770 / 10110 | 2020 / 1900 | 870 / 2260 | 920 / 2370 | 660 / 2590 |
| 5,000, 2,000 | 270 / 250 | 210 / 200 | 140 / 280 | 120 / 310 | 120 / 360 |
alpha.bst at 50,000 went from 10730 ms to 2080 ms at one thread and 560 ms at
18. The .bbl and the warnings are byte-identical to the previous build at
every thread count, and match bibtex 0.99e for plain, alpha and unsrt.
abbrv still differs from bibtex where it did before, on a braced First part
such as {van der}, which bibtex abbreviates to v..
Cross-reference pass. A LaTeX document that both labels and refers (or
cites and has a bibliography) is lowered twice per run: once by the pass that
collects its labels into the .aux, once to typeset against the .aux. The
pass reads only the source, so when an .aux from an earlier run is on disk
the two run concurrently, the typesetting against the .aux as it stands; when
the pass is done its .aux is written as before, and if that file would parse
differently from the old one the lowering is redone against the new one
(latex::aux::update_then). Package reports on stderr are held on both sides
and released in sequential order. It stays sequential at one thread, on a
first run, and for documents using Lua, \rust blocks or URL inputs.
Measured 2026-10-17 on a 1001-page article with 2000 labels and 2000
references (load averages 18 to 30, medians of 15 interleaved runs, default
thread count): an unchanged rerun went from 1146 ms to 686 ms wall, user CPU
2022 to 2087 ms, peak RSS about 205 to 335 MB; a run after an edit that
renumbers every label went from 1000 ms to 1123 ms wall (2005 to 2637 ms user),
the cost of a lowering thrown away. stdout, stderr, the PDF and the .aux are
byte-identical to the previous build and across thread counts.
Iteration depth. \def\r{... \ifnum ... \r \fi} is TeX's loop, and
tex recurses through it — one input-stack level per turn, giving up at
[input stack size=10000]. texrs lowers that shape to a backward jump, so its
stack use is constant and the bound is arithmetic rather than memory:
| iterations | 9,000 | 12,000 | 1,000,000 |
|---|---|---|---|
tex |
ok | capacity exceeded | capacity exceeded |
| texrs | ok | ok | ok |
Measured, 2026-10-17 at 0d7158a91a
Taken at commit 0d7158a91a, release build, on an 18-core aarch64 macOS machine
that was under other load the whole time (1-minute load average between 15 and
26). Every figure is a median of interleaved runs — 9 per thread count for the
large document, 7 for the other two — with a warm kpse path cache and
SOURCE_DATE_EPOCH=1, on the default PDF path. The stage split comes from
timers placed between stages in a scratch build; they are not in the tree.
Where one document's time goes: a 1001-page LaTeX document from 4.3 MB of source, wall time in ms.
| stage | 1 thread | 8 threads |
|---|---|---|
| process start and exit | 8 | 8 |
| argument parsing, reading the source | 8 | 9 |
desugar, .aux update |
3 | 3 |
preamble text (latex::preamble_at) |
100 | 102 |
| preamble preload | 3 | 3 |
lowering (Lowerer::lower) |
190 | 194 |
| compile and VM run | 4 | 4 |
| font setup, references, contents | 1 | 1 |
line breaking (typeset::break_lines_measured) |
394 | 408 |
pagination (typeset::paginate) |
39 | 39 |
page-start prescan (typeset::page_starts) |
24 | 24 |
| page drawing (parallel) | 419 | 79 |
pdf::document |
48 | 51 |
| page count of the written PDF, twice | 43 | 45 |
| file write | 7 | 17 |
| drops and returns | 3 | 4 |
| sum of stage medians | 1294 | 990 |
| median wall | 1351 | 990 |
| median user CPU | 1250 | 1430 |
The kpse startup prefetch runs on a thread of its own and overlaps the stages after it, so it has no row. At one thread the pages are drawn on the pool's one worker while the calling thread waits, which is the drawing row.
Page drawing is the only stage that fans out. At 8 threads the other ~910 ms is serial, and most of it is in three places:
- Line breaking,
typeset::break_lines_measured: one loop over paragraphs carrying the colour, face and size stacks, the centring flag and the list depth. A paragraph'sfilldepends on nothing else, so it can be split the way drawing was — a serial pass recording that state per paragraph, then the paragraphs in parallel. About 40% of it islinebreak::Hyphenator::points. Since this measurement it is split that way and the patterns are a trie; see Line breaking above. - Lowering,
lower::Lowerer::lower: the expander, whose macro and register state carries from token to token. Serial by nature. Since this measurement the paragraphs it has finished are broken on the pool while it lowers the rest; see Line breaking above. - Preamble text,
latex::preamble_at: package loading (load::preamble, ~42 ms), the counter scan inlatex::support(~22 ms), the class check innumbering(~22 ms) and the label seeds (~8 ms) — independent scans of the same source. Since this measurement the class check and both label seedings run on threads of their own beside the package load, andsupport's four scans run side by side, so only the load andsupportare left in series.
Smaller: pdf::document spends most of its time in the SHA-256 of the whole
file for /ID (pdf::file_identifier), hashed in software on aarch64 at the
time (sha2's asm feature, which uses the ARMv8 SHA-2 instructions, is on
since, and the /ID has since become luatex's, which hashes no file at all); the page count was taken by inflating
the finished PDF, once in main.rs write_pdf and again in
pdf_output_line (since this measurement it is the writer's own count,
carried beside the bytes in texrs::Pdf, and nothing is inflated);
paginate is sequential by definition (since this measurement page_starts
reads each page in parallel and only applies the per-page effects in series,
and paginate gathers its page-break offers, penalties and leadings in
parallel; see Page breaking above).
Scaling with TEXRS_THREADS, median wall in ms:
| document | 1 | 2 | 4 | 8 | 16 |
|---|---|---|---|---|---|
| LaTeX, 1001 pages, 4.3 MB | 1351 | 1188 | 1097 | 990 | 933 |
| plain TeX, 394 pages, 2.0 MB | 425 | 383 | 343 | 310 | 302 |
| LaTeX, 200 pages, 0.9 MB | 271 | 240 | 221 | 211 | 209 |
[0x0B] Releasing
The version lives in seven tracked files — the manifest, two hand-written docs
pages, the generated reference page, two man pages, and the IntelliJ plugin's
gradle.properties. (tests/version_sync.rs reads all seven; the count in this
sentence said six while the list beside it named seven.) Nothing in a build or a test run notices when they
disagree, which is how v0.1.0 once sat in the docs through v0.3.0. So
tests/version_sync.rs fails when any of them drifts, and scripts/bump.sh is
the one command that stamps all seven, regenerates the two that are derived from
the corpus rather than substituted, runs the full verify, then tags, pushes and
publishes.
[0x0C] Documentation
- Docs hub — menketechnologies.github.io/texrs (
docs/index.html) - Engineering report — architecture, what lowering forces, parity posture, dependencies (
docs/report.html) - Primitive reference — every primitive texrs carries and where it happens (
docs/reference.html, generated fromsrc/corpus.rswithcargo run --bin gen-docs) - Known gaps — the ledger, each entry pinned by a case the suite gates on (
BUGS.md) - Roadmap — the milestones, and what the stomach still lacks (
docs/ROADMAP.md)
[0xFF] Licence
MIT.