Expand description
texrs — a TeX engine.
Milestone 1 is Knuth’s MOUTH and EXPANDER: category codes, the three-state
line scanner, \def with delimited parameters, \csname, \the, the count
registers and their arithmetic, and \message. That is the half of TeX that
produces tokens rather than boxes, and it is the half a macro-heavy document
spends its time in.
It is NOT a typesetter yet: there are no boxes, no glue, no paragraphs and no
DVI. docs/ROADMAP.md says what that would take. The parity contract for
this milestone is the \message stream, compared byte-for-byte against real
tex by cargo run --bin parity.
Re-exports§
pub use embed::run_program;pub use expand::Engine;pub use expand::TexError;
Modules§
- agl
- Glyph names to Unicode, ported from
agl.cin tectonic’sxdvipdfmx. - aot
- Ahead-of-time compilation: a TeX document as a standalone native binary.
- aot_
runtime - The runtime a standalone AOT binary links: the builtins hook, the entry, and the page inputs embedded beside the compiled chunk.
- backend
- LuaTeX’s PDF backend as a document reaches it:
\pdfvariable,\pdffeedbackand\pdfextension(textoken.c’sdo_variable_pdfanddo_feedback_pdf,extensions.c’sdo_extension_pdf, and the scanners inpdfobj.c,pdfannot.c,pdfdest.c,pdflink.c,pdfoutline.c,pdfaction.c,pdfthread.candwriteimg.c). - banner
- The texrs logo, the live-stats box, and the version line.
- bib
- Reading a
.bibdatabase, ported from the reading half of tectonic’sengine_bibtex. - box_
- The boxes a document can nest:
\hbox,\vbox,\vtop, the rules, the shifts, the unboxing, and the glue and kerns that go between them. - bst
- Reading a
.bststyle, ported from the front of tectonic’sengine_bibtex. - bstvm
- Running a
.bst: the other half of BibTeX, ported frombibtex.web. - bundle
- A bundle: a document’s support files as one archive, ported from the local
half of tectonic’s
bundles. - catcode
- TeX’s category codes — the sixteen classes every character belongs to.
- cff
- Reading the
CFFtable, ported fromcff.candcff_dict.cin tectonic’sxdvipdfmx. - charcodes
- TeX’s other per-character tables.
- cli
- The command line, in
tex’s own grammar. - colour
- The colours a document names, and the models it names them in.
- compiler
- Lower the command stream to fusevm bytecode.
- corpus
- The primitive-reference corpus: the
(name, chapter, doc, syntax + example)table behind editor completion, editor hover (src/lsp.rs) and the generateddocs/reference.html(src/bin/gen_docs.rs). - dap
- Debug Adapter Protocol over stdio (
texrs --dap). - dimen
- Dimensions: scaled points, the units that reach them, and how TeX prints one back.
- docs
docs/reference.html, rendered from the primitive corpus.- document
Texrs.toml: a document that is more than one file, ported in shape from tectonic’sdocmodel.- dvi
- Reading DVI, ported from tectonic’s
xdv. - dvi_
parity - DVI parity: the same document through
texand through texrs. - embed
- The embedding entry point: run a whole document with a given terminal input and get back its text.
- etex_
arith - The arithmetic of e-TeX’s
\numexprand\dimexpr, as luatex’sscanning.cdoes it (add_or_sub,quotient,fract, and tex.web §105’smult_and_addbehindmult_integersandnx_plus_y). - expand
- TeX’s expander: the half of the engine that turns tokens into other tokens.
- fontmap
- Reading a font map and an encoding file, ported from
fontmap.candt1_load_encin tectonic’sxdvipdfmx. - fontreq
- What a
\fontrequest means to luaotfload. - format
- A format: the engine as a preamble left it, saved so the next run starts
there. Ported in shape from tectonic’s
xetex_format. - geturl
- Fetching a bundle, ported from tectonic’s
geturl. - glue
- Glue: a dimension that can stretch and shrink.
- glyf
- TrueType outlines and cutting a font down to the glyphs a document used,
ported from
tt_glyf.cin tectonic’sxdvipdfmx. - image
- Reading an image’s header, ported from
pngimage.candjpegimage.cin tectonic’sxdvipdfmx. - include
- Putting a page of one PDF onto a page of another, ported from
pdf_include_pagein tectonic’sxdvipdfmx. - input_
hooks - Where LuaTeX’s input callbacks reach the engine:
find_read_file,open_read_fileandprocess_input_buffer(the LuaTeX manual, “Callbacks”, “File discovery” and “Data processing”). - intercepts
- Advice on macro expansion: before, after and around, matched by glob.
- intpar
- TeX’s integer parameters (
tex.web§236), as frontend state. - io
- Where a document’s bytes come from, ported in shape from Tectonic’s
io_base. - ir
- The command stream a TeX file compiles to.
- itar
- An indexed tar bundle, ported from
tectonic_bundles::itar. - known
- What the frontend knows of the registers while it is lowering.
- kpse
- File location through
kpsewhich, shared by every caller. - latex
- The LaTeX layer texrs can honour.
- lexer
- TeX’s mouth: bytes to tokens.
- linebreak
- Breaking a paragraph the way
tex.web§813 does: by minimising the total demerits of the WHOLE paragraph, with Liang hyphenation to widen the set of places a line may end. - lower
- Turn a TeX source into the command stream
crate::compilerlowers. - lsp
- Language Server Protocol over stdio (
texrs --lsp). - lua
\directlua: the Lua a LuaTeX document contains, actually run.- luatex_
oracle - The second oracle: what
luatexsays, for the primitives LuaTeX has and TeX82 has not. - luatex_
primitives - Every primitive a document can reach in luatex 1.24.0, with the command
code luatex gives it (
token.create(name).cmdname) and whether that command expands. WRITTEN bybash scripts/primitive-census.shout of luatex’s own table (crate::luatex_oracle::command_table); not edited by hand, andtests/luatex_primitives.rschecks it against luatex. - math
- Mathematics:
$…$,$$…$$, and whattex.webdoes between them. - node
- The node list: what a TeX list is made of, once the mouth and the expander have finished with it.
- otf
- OpenType fonts as luaotfload’s node mode loads them.
- pack
- Packaging:
hpack,vpack, badness, and the glue setting that makes a box come out at the width it was asked for. - page
- Breaking vertical lists into pages:
\vsplit, the page builder, insertions, marks and the output routine. - parallel
- Lexing TeX in parallel.
- params
- TeX’s dimension, glue and token parameters (
tex.web§247, §224, §230). - parbreak
- Breaking a paragraph’s node list into lines:
tex.web§813-§876. - parity
- The oracle: what real
texsays about a document, and whether we agree. - Writing PDF, ported from
pdfobj.candpdfdoc.cin tectonic’sxdvipdfmx. - pdf_
parity - PDF parity: the same document through
luatexand through texrs, compared as far as they agree. - pdfdoc
- The file a document’s pages go into, laid out as LuaTeX lays it out: a port
of the object machinery of
pdfgen.c(pdf_create_obj,pdf_begin_obj, the object streams,pdf_dict_add_streaminfo,pdf_finish_file), ofpdfpagetree.c, and of the orderwritefont.cwrites a font’s objects in. - pdfpage
- A page’s content stream, written as LuaTeX’s PDF backend writes it: a port
of
pdfpage.c,pdfglyph.c,pdfrule.c,pdfliteral.c,pdfsaverestore.candpdfsetmatrix.cfromluatexdir/pdf. - pdfread
- Reading a PDF, ported from
pdfparse.cand the reading half ofpdfobj.cin tectonic’sxdvipdfmx. - pdfship
- Shipping a box to a PDF page, as LuaTeX does it:
pdfshipout.c’sship_outandpdflistout.c’shlist_outandvlist_out, for boxes in the defaultTLTdirection. - pk
- Reading a
.pk, the packed bitmap font, ported frompkfont.cin tectonic’sxdvipdfmx. - postline
- Turning chosen breakpoints into lines:
post_line_break. - primitives
- TeX82’s primitive control sequences, as
\meaningand\showname them. - repl
- Interactive prompt for
texrs --repl. - runtime
- Run a compiled chunk on fusevm.
- rust_
ffi - Inline Rust in a TeX document:
\rust{ ... }blocks. - script_
cache - rkyv-backed bytecode cache for
.texfiles. - sfnt
- Reading an OpenType or TrueType font, ported from
sfnt.c,tt_table.c,tt_cmap.candtt_post.cin tectonic’sxdvipdfmx. - shape
- OpenType shaping as LuaHBTeX’s
mode=harfdoes it. - shipout
- Drawing a box tree into a DVI file:
tex.web§619-§640. - special
- Reading a
\special, ported from thespc_*.cfamily in tectonic’sxdvipdfmx. - status
- Where a diagnostic goes, ported in shape from Tectonic’s
status_base. - stomach
- The stomach at run time: the current list, the box registers, the fonts
and the page, acted on as
tex.web’smain_controlacts on them. - tfm
- Reading a
.tfm, the font metrics every TeX engine measures with. - tiers
- Which fusevm execution tier a program’s bytecode actually reaches.
- tikz
- TikZ: the picture language, as far as a document draws with it.
- token
- What the mouth produces: TeX has exactly two kinds of token.
- type1
- Reading a Type 1 font, ported from
type1.candt1_char.cin tectonic’sxdvipdfmx. - typeset
- The smallest honest stomach: text and font metrics into DVI pages.
- vf
- Reading a
.vf, the virtual font a driver has to unpack before it can set a character. - write
\openout,\write,\closeoutand\special:tex.web§1341-§1378 as LuaTeX extends them inextensions.candtexfileio.c.
Structs§
- A finished PDF and the number of pages the writer put in it.
- PdfProgram
- The bytecode of a PDF run and what the page builder reads beside it: the
part of
run_pdf_with_messagesthat is fixed once the document is lowered.
Functions§
- commands
- The command stream
srclowers to, for--dump-astand for tests that want to read the frontend’s output before the code generator touches it. - compile
- The bytecode
srccompiles to, for--disasmand for tests that want to see that a construct really lowered rather than being folded away. - compile_
cached - The bytecode for the document at
path, from the cache when the file has not changed since it was compiled, and put there when it has. - compile_
debug - Compile
srcwith the--dapstatement markers in it. - compile_
located - Compile
src, reporting the line the mouth had reached if it stopped. - compile_
pdf_ program PdfProgramforsrc, read the wayrun_pdf_with_messagesreads it: desugared, the LaTeX preamble preloaded, and\refanswered from the.auxbesidepathafter the label pass has brought it up to date.- compile_
text - The bytecode
run_textruns: the same pipeline, with the document’s own words lowered as well as its messages. - file_
line - The
(./doc.tex …)line tex prints for a file. - pdf_
output_ line - What a run says about the PDF it produced, the way lualatex says it.
- pdf_
page_ count - How many pages a PDF declares.
- run_dvi
- run_
dvi_ cached - run_
dvi_ fallback - run_
list_ path - Run
srcon the list path – paragraphs, the page builder and\outputall in the stomach (crate::stomach) – and hand back the message stream and the stomach it left, with its shipped pages and the fonts they are set in. - run_
messages - run_
messages_ cached - The same, for a document read from
path: the bytecode comes from the cache when the file has not changed since it was compiled, and is put there when it has. The result is identical either way — the cache is a way of skipping the front of the pipeline, never of changing what it produces. - run_
messages_ debug - Run a document under the debug adapter: markers installed, no tracing JIT.
- run_
messages_ initex - What an INITEX run prints for
src: no format preloaded, every table as §240 leaves it – the stateluatex --inistarts from, which is whatcrate::luatex_oraclecompares against. - run_
messages_ list - The messages
srcwrites, as a list rather than one joined line. - run_
messages_ tex - The
\messagestream a run produces when it is meant to read like tex’s: an undefined control sequence is reported on the terminal and expands to nothing, as §370 has it, whererun_messageshands an embedder theErrit asked for. A LaTeX document keeps the stop either way, because the loader reads that stop as “this package needs a macro texrs lacks”. - run_pdf
- Typeset a file to DVI, using the bytecode cache. Typeset with a font CHAIN: the primary font, and fallbacks for the glyphs it does not carry.
- run_
pdf_ at - The same, told which FILE the document was read from, so a path it names can be resolved relative to the directory holding it.
- run_
pdf_ with_ messages - The same, keeping the
\messagestream the run produced. - run_
tex_ dvi - What
tex FILEdoes: plain.tex as the preloaded format, then the document, on the list path – and the DVI file the shipped pages make (Nonewhen none was shipped), with the message stream. - run_
tex_ pdf - What
luatex FILEdoes with a plain document: plain.tex as the format, the document on the list path, and each shipped page drawn as LuaTeX’s backend draws it (crate::pdfship::document) – with the message stream, andNonewhen no page was shipped. The page is\pagewidthby\pageheightwith the reference point at\pdfvariable horiginandvorigin, and where the document set none of them,luatex.fmt’s: 8.5in by 11in, an inch in from the top left. - run_
text - run_
text_ at - The document’s text, told which FILE it was read from and not using the bytecode cache.
- run_
text_ cached - Typeset a document to DVI, and say where the font came from.
- run_
text_ marked - The document’s text WITH the colour markers, for the typesetter.
- run_
text_ marked_ cached - The same, keeping the colour markers, for the typesetter.
- source_
ends_ run - Whether an
\endinsrcstops the run, rather than the source merely running out. - text_
without_ marks - The document’s text with every typesetting marker taken out.