Skip to main content

Crate texrs

Crate texrs 

Source
Expand description

texrs — a TeX engine.

Milestone 1 is Knuth’s MOUTH and EXPANDER: category codes, the three-state line scanner, \def with delimited parameters, \csname, \the, the count registers and their arithmetic, and \message. That is the half of TeX that produces tokens rather than boxes, and it is the half a macro-heavy document spends its time in.

It is NOT a typesetter yet: there are no boxes, no glue, no paragraphs and no DVI. docs/ROADMAP.md says what that would take. The parity contract for this milestone is the \message stream, compared byte-for-byte against real tex by cargo run --bin parity.

Re-exports§

pub use embed::run_program;
pub use expand::Engine;
pub use expand::TexError;

Modules§

agl
Glyph names to Unicode, ported from agl.c in tectonic’s xdvipdfmx.
aot
Ahead-of-time compilation: a TeX document as a standalone native binary.
aot_runtime
The runtime a standalone AOT binary links: the builtins hook, the entry, and the page inputs embedded beside the compiled chunk.
backend
LuaTeX’s PDF backend as a document reaches it: \pdfvariable, \pdffeedback and \pdfextension (textoken.c’s do_variable_pdf and do_feedback_pdf, extensions.c’s do_extension_pdf, and the scanners in pdfobj.c, pdfannot.c, pdfdest.c, pdflink.c, pdfoutline.c, pdfaction.c, pdfthread.c and writeimg.c).
banner
The texrs logo, the live-stats box, and the version line.
bib
Reading a .bib database, ported from the reading half of tectonic’s engine_bibtex.
box_
The boxes a document can nest: \hbox, \vbox, \vtop, the rules, the shifts, the unboxing, and the glue and kerns that go between them.
bst
Reading a .bst style, ported from the front of tectonic’s engine_bibtex.
bstvm
Running a .bst: the other half of BibTeX, ported from bibtex.web.
bundle
A bundle: a document’s support files as one archive, ported from the local half of tectonic’s bundles.
catcode
TeX’s category codes — the sixteen classes every character belongs to.
cff
Reading the CFF table, ported from cff.c and cff_dict.c in tectonic’s xdvipdfmx.
charcodes
TeX’s other per-character tables.
cli
The command line, in tex’s own grammar.
colour
The colours a document names, and the models it names them in.
compiler
Lower the command stream to fusevm bytecode.
corpus
The primitive-reference corpus: the (name, chapter, doc, syntax + example) table behind editor completion, editor hover (src/lsp.rs) and the generated docs/reference.html (src/bin/gen_docs.rs).
dap
Debug Adapter Protocol over stdio (texrs --dap).
dimen
Dimensions: scaled points, the units that reach them, and how TeX prints one back.
docs
docs/reference.html, rendered from the primitive corpus.
document
Texrs.toml: a document that is more than one file, ported in shape from tectonic’s docmodel.
dvi
Reading DVI, ported from tectonic’s xdv.
dvi_parity
DVI parity: the same document through tex and through texrs.
embed
The embedding entry point: run a whole document with a given terminal input and get back its text.
etex_arith
The arithmetic of e-TeX’s \numexpr and \dimexpr, as luatex’s scanning.c does it (add_or_sub, quotient, fract, and tex.web §105’s mult_and_add behind mult_integers and nx_plus_y).
expand
TeX’s expander: the half of the engine that turns tokens into other tokens.
fontmap
Reading a font map and an encoding file, ported from fontmap.c and t1_load_enc in tectonic’s xdvipdfmx.
fontreq
What a \font request means to luaotfload.
format
A format: the engine as a preamble left it, saved so the next run starts there. Ported in shape from tectonic’s xetex_format.
geturl
Fetching a bundle, ported from tectonic’s geturl.
glue
Glue: a dimension that can stretch and shrink.
glyf
TrueType outlines and cutting a font down to the glyphs a document used, ported from tt_glyf.c in tectonic’s xdvipdfmx.
image
Reading an image’s header, ported from pngimage.c and jpegimage.c in tectonic’s xdvipdfmx.
include
Putting a page of one PDF onto a page of another, ported from pdf_include_page in tectonic’s xdvipdfmx.
input_hooks
Where LuaTeX’s input callbacks reach the engine: find_read_file, open_read_file and process_input_buffer (the LuaTeX manual, “Callbacks”, “File discovery” and “Data processing”).
intercepts
Advice on macro expansion: before, after and around, matched by glob.
intpar
TeX’s integer parameters (tex.web §236), as frontend state.
io
Where a document’s bytes come from, ported in shape from Tectonic’s io_base.
ir
The command stream a TeX file compiles to.
itar
An indexed tar bundle, ported from tectonic_bundles::itar.
known
What the frontend knows of the registers while it is lowering.
kpse
File location through kpsewhich, shared by every caller.
latex
The LaTeX layer texrs can honour.
lexer
TeX’s mouth: bytes to tokens.
linebreak
Breaking a paragraph the way tex.web §813 does: by minimising the total demerits of the WHOLE paragraph, with Liang hyphenation to widen the set of places a line may end.
lower
Turn a TeX source into the command stream crate::compiler lowers.
lsp
Language Server Protocol over stdio (texrs --lsp).
lua
\directlua: the Lua a LuaTeX document contains, actually run.
luatex_oracle
The second oracle: what luatex says, for the primitives LuaTeX has and TeX82 has not.
luatex_primitives
Every primitive a document can reach in luatex 1.24.0, with the command code luatex gives it (token.create(name).cmdname) and whether that command expands. WRITTEN by bash scripts/primitive-census.sh out of luatex’s own table (crate::luatex_oracle::command_table); not edited by hand, and tests/luatex_primitives.rs checks it against luatex.
math
Mathematics: $…$, $$…$$, and what tex.web does between them.
node
The node list: what a TeX list is made of, once the mouth and the expander have finished with it.
otf
OpenType fonts as luaotfload’s node mode loads them.
pack
Packaging: hpack, vpack, badness, and the glue setting that makes a box come out at the width it was asked for.
page
Breaking vertical lists into pages: \vsplit, the page builder, insertions, marks and the output routine.
parallel
Lexing TeX in parallel.
params
TeX’s dimension, glue and token parameters (tex.web §247, §224, §230).
parbreak
Breaking a paragraph’s node list into lines: tex.web §813-§876.
parity
The oracle: what real tex says about a document, and whether we agree.
pdf
Writing PDF, ported from pdfobj.c and pdfdoc.c in tectonic’s xdvipdfmx.
pdf_parity
PDF parity: the same document through luatex and through texrs, compared as far as they agree.
pdfdoc
The file a document’s pages go into, laid out as LuaTeX lays it out: a port of the object machinery of pdfgen.c (pdf_create_obj, pdf_begin_obj, the object streams, pdf_dict_add_streaminfo, pdf_finish_file), of pdfpagetree.c, and of the order writefont.c writes a font’s objects in.
pdfpage
A page’s content stream, written as LuaTeX’s PDF backend writes it: a port of pdfpage.c, pdfglyph.c, pdfrule.c, pdfliteral.c, pdfsaverestore.c and pdfsetmatrix.c from luatexdir/pdf.
pdfread
Reading a PDF, ported from pdfparse.c and the reading half of pdfobj.c in tectonic’s xdvipdfmx.
pdfship
Shipping a box to a PDF page, as LuaTeX does it: pdfshipout.c’s ship_out and pdflistout.c’s hlist_out and vlist_out, for boxes in the default TLT direction.
pk
Reading a .pk, the packed bitmap font, ported from pkfont.c in tectonic’s xdvipdfmx.
postline
Turning chosen breakpoints into lines: post_line_break.
primitives
TeX82’s primitive control sequences, as \meaning and \show name them.
repl
Interactive prompt for texrs --repl.
runtime
Run a compiled chunk on fusevm.
rust_ffi
Inline Rust in a TeX document: \rust{ ... } blocks.
script_cache
rkyv-backed bytecode cache for .tex files.
sfnt
Reading an OpenType or TrueType font, ported from sfnt.c, tt_table.c, tt_cmap.c and tt_post.c in tectonic’s xdvipdfmx.
shape
OpenType shaping as LuaHBTeX’s mode=harf does it.
shipout
Drawing a box tree into a DVI file: tex.web §619-§640.
special
Reading a \special, ported from the spc_*.c family in tectonic’s xdvipdfmx.
status
Where a diagnostic goes, ported in shape from Tectonic’s status_base.
stomach
The stomach at run time: the current list, the box registers, the fonts and the page, acted on as tex.web’s main_control acts on them.
tfm
Reading a .tfm, the font metrics every TeX engine measures with.
tiers
Which fusevm execution tier a program’s bytecode actually reaches.
tikz
TikZ: the picture language, as far as a document draws with it.
token
What the mouth produces: TeX has exactly two kinds of token.
type1
Reading a Type 1 font, ported from type1.c and t1_char.c in tectonic’s xdvipdfmx.
typeset
The smallest honest stomach: text and font metrics into DVI pages.
vf
Reading a .vf, the virtual font a driver has to unpack before it can set a character.
write
\openout, \write, \closeout and \special: tex.web §1341-§1378 as LuaTeX extends them in extensions.c and texfileio.c.

Structs§

Pdf
A finished PDF and the number of pages the writer put in it.
PdfProgram
The bytecode of a PDF run and what the page builder reads beside it: the part of run_pdf_with_messages that is fixed once the document is lowered.

Functions§

commands
The command stream src lowers to, for --dump-ast and for tests that want to read the frontend’s output before the code generator touches it.
compile
The bytecode src compiles to, for --disasm and for tests that want to see that a construct really lowered rather than being folded away.
compile_cached
The bytecode for the document at path, from the cache when the file has not changed since it was compiled, and put there when it has.
compile_debug
Compile src with the --dap statement markers in it.
compile_located
Compile src, reporting the line the mouth had reached if it stopped.
compile_pdf_program
PdfProgram for src, read the way run_pdf_with_messages reads it: desugared, the LaTeX preamble preloaded, and \ref answered from the .aux beside path after the label pass has brought it up to date.
compile_text
The bytecode run_text runs: the same pipeline, with the document’s own words lowered as well as its messages.
file_line
The (./doc.tex …) line tex prints for a file.
pdf_output_line
What a run says about the PDF it produced, the way lualatex says it.
pdf_page_count
How many pages a PDF declares.
run_dvi
run_dvi_cached
run_dvi_fallback
run_list_path
Run src on the list path – paragraphs, the page builder and \output all in the stomach (crate::stomach) – and hand back the message stream and the stomach it left, with its shipped pages and the fonts they are set in.
run_messages
run_messages_cached
The same, for a document read from path: the bytecode comes from the cache when the file has not changed since it was compiled, and is put there when it has. The result is identical either way — the cache is a way of skipping the front of the pipeline, never of changing what it produces.
run_messages_debug
Run a document under the debug adapter: markers installed, no tracing JIT.
run_messages_initex
What an INITEX run prints for src: no format preloaded, every table as §240 leaves it – the state luatex --ini starts from, which is what crate::luatex_oracle compares against.
run_messages_list
The messages src writes, as a list rather than one joined line.
run_messages_tex
The \message stream a run produces when it is meant to read like tex’s: an undefined control sequence is reported on the terminal and expands to nothing, as §370 has it, where run_messages hands an embedder the Err it asked for. A LaTeX document keeps the stop either way, because the loader reads that stop as “this package needs a macro texrs lacks”.
run_pdf
Typeset a file to DVI, using the bytecode cache. Typeset with a font CHAIN: the primary font, and fallbacks for the glyphs it does not carry.
run_pdf_at
The same, told which FILE the document was read from, so a path it names can be resolved relative to the directory holding it.
run_pdf_with_messages
The same, keeping the \message stream the run produced.
run_tex_dvi
What tex FILE does: plain.tex as the preloaded format, then the document, on the list path – and the DVI file the shipped pages make (None when none was shipped), with the message stream.
run_tex_pdf
What luatex FILE does with a plain document: plain.tex as the format, the document on the list path, and each shipped page drawn as LuaTeX’s backend draws it (crate::pdfship::document) – with the message stream, and None when no page was shipped. The page is \pagewidth by \pageheight with the reference point at \pdfvariable horigin and vorigin, and where the document set none of them, luatex.fmt’s: 8.5in by 11in, an inch in from the top left.
run_text
run_text_at
The document’s text, told which FILE it was read from and not using the bytecode cache.
run_text_cached
Typeset a document to DVI, and say where the font came from.
run_text_marked
The document’s text WITH the colour markers, for the typesetter.
run_text_marked_cached
The same, keeping the colour markers, for the typesetter.
source_ends_run
Whether an \end in src stops the run, rather than the source merely running out.
text_without_marks
The document’s text with every typesetting marker taken out.