htl-core 0.3.0

Engine of htl (Holistic Typed Lua): Teal check / gen / lint / fmt / test / bundle on an embedded mlua state. Use the `htl` crate.
Documentation

htl — Holistic Typed Lua

Teal (typed Lua) with the toolchain hidden behind cargo. One binary type-checks, lints, formats, tests, bundles and runs .tl; one proc macro makes Teal type errors fail cargo build; one resolver puts .tl modules into mlua-pkg's require chain. The Teal compiler (tl.lua) is embedded in the mlua state — there is no luarocks, no tl CLI, no generated .lua in your tree.

scripts/foo.tl ──include_tl!──▶ cargo build   (Teal type error = rustc error, with span)
               ──htl run ─────▶ check → gen → load, in one mlua state
               ──htl build────▶ stripped Lua 5.4 bytecode bundle (.hb), no source shipped
Rust impl Host ──#[host_module]▶ UserData impl + host.d.tl   (Rust signature change breaks .tl at build)

Install

cargo install htl-cli          # binaries: htl, cargo-htl  (so `cargo htl <verb>` works)

[dependencies]
htl = "0.1"                    # embedding: engine + proc macros in one import
crate role
htl umbrella: re-exports htl-core and (feature macros, default on) the proc macros. Depend on this one.
htl-core engine: Htl, lints, fmt, bundle, test runner, mlua-pkg resolver
htl-macros include_tl! / include_tl_bytes! / TealRecord / host_module; generated code targets ::htl::
htl-cli the htl / cargo-htl binaries

CLI

command what it does
htl new <name> / htl init [dir] scaffold: mlua-pkg.toml, src/<mod>/init.tl, src/main.tl, tests/, README (--lib, --embed for a Rust host)
htl check [paths] [--strict] [--lint +rule,-rule] [--no-cache] [--cache-mode per-module|whole-run] [--explain-cache] type-check; htl lints as lint: (advisory, --strict fails on them); a module reached through require (an installed dep, a [check] paths dir) is checked with the file and its type errors are errors too, once per run, with the file that required it; what has not changed is replayed from .htl/ (see Caching)
htl run <file.tl | app.hb> [args] check then execute; require of a .tl with type errors fails
htl test [paths] [--filter s] [--lib mod] [--coverage] [--lcov file] [--no-cache] *_test.tl and tests/**/*.tl, one isolated state per file; checking is replayed from .htl/, the run never is (see Caching)
htl fmt [paths] [--check] [--indent N] whitespace formatter (indentation from the syntax tree, blank lines, trailing space)
htl gen <file.tl> [-o out.lua] readable Lua, the escape hatch out of htl
htl build <entry.tl> -o app.hb [--debug] [--source] [--extra a,b] [--host x,y] link the entry's require closure into one bundle (see Bundles)
htl bundle info <app.hb> [--format json] what a bundle records, without running it: format, the htl that built it, payload kind, the Lua its bytecode is for, entry, modules, host-provided names
htl pkg install fetch every dependency mlua-pkg.toml declares into .htl/modules/ and write mlua-pkg.lock; the deps' own types/ are then copied into the project's (see types/)
htl pkg add <name> <git> [--tag t | --rev r | --branch b] [--entry dir] [--target-dir dir] write the dependency into the manifest (install fetches it); a patch_dir the entry already declared is kept
htl pkg update [name] [--dry-run] [--force] refresh dependencies and bump the pins that follow releases, then install
htl pkg clean [--all] remove cached packages the lockfile no longer refers to, or the whole cache
htl pkg patch <dep> [--force] take that dependency's source into patches/<dep>/, where the project owns it and install resolves it from (see Patched dependencies)
htl types add <library> [--from dir] [--force] the declarations a library never shipped, from teal-types, into types/ with the commit they came from recorded beside each
htl cache status [path] [--entries] / htl cache clear [path] report what the store holds, or empty it (see Caching)
htl dts [dir] write the .d.tl files this project declares: from Rust source, the ones #[host_module] / #[derive(TealRecord)] ask for, no build needed; from Teal, the module each ---@contract type is declared in. check / run / test / build do this automatically; exits non-zero when something it was asked to write could not be

mlua-pkg.toml is detected by walking up from the file: installed deps become visible to the checker and to run / test / build automatically. They go under .htl/modules/, beside the check cache — htl decides that one location, and the installer is mlua-pkg's library rather than its binary, so there is no second process to agree with and nothing on PATH to install. MLUA_PKG_DIR and a target/ in the working directory, which the mlua-pkg binary reads, are not consulted. Vendored is kept for the other thing: a copy of a dependency committed to the repo, which a target_dir entry in the manifest declares and nothing does by default. When a directory is given, check / fmt / build / test walk the project's own files only: target/, node_modules/, .mlua-pkgs/ and any dot-directory are not entered, so dependencies' sources and tests stay theirs. A directory passed explicitly is always walked. A target_dir copy is not entered either, and there the manifest is what says so: the copy sits in the repo under a name the project chose, so nothing about the path tells it apart from the project's own code beside it. mlua-pkg install rewrites it every time it runs — checking it would report a dependency's errors as the project's, htl fmt would write a diff against upstream that the next install undoes, and its *_test.tl are a dependency's suite (Go's ./... has excluded vendor/ since 1.9 for the same reason). A patch_dir dependency is the one thing in between: check reads it, fmt and test do not (see Patched dependencies). What is not walked is still checked: a dependency is checked through the require that reaches it, and a type error in it is reported as an error with the dependency's own path and the file that required it —

error: .htl/modules/vendored/mathx/init.tl:12:8: in local declaration: got string, expected number
  (required by src/geometry.tl)

— once per run however many files require it, and replayed from the cache like the requirer's own diagnostics. Paths read against the directory the command ran in, whether the walk or a require found the file, and one that lies outside it is written in full rather than as a stack of ... htl run would refuse the module at that require; the check says so first. Files under tests/ are checked with the project root and src/ on the search path, the same as htl test, so htl check tests and htl test agree.

Caching

htl check stores what it worked out under .htl/cache/ at the project root and replays whatever has not moved. The summary says how much: [cached] when everything came from the store and no checker was built at all, [36/48 cached] when some of it did, and nothing when none did. --format json carries the same as summary.cached and summary.replayed. htl init puts .htl/ in .gitignore — one line for the cache and the installed deps beside it; add it by hand in an existing project.

There are two separate controls. Whether to cache is --no-cache, which neither reads nor writes. How the cache is grained is --cache-mode, or [cache] mode in htl.toml with the flag overriding it:

mode entry an edit costs
per-module (default) one per module that module, whatever requires it, and what those pull in
whole-run one for the walk the whole walk, wherever the edit landed

Measured on a 57-module project of about 14,000 lines, release build, wall clock:

per-module whole-run
cold (--no-cache) 1.81 s 1.81 s
nothing edited 0.018 s 0.017 s
a module nothing requires 0.33 s 1.75 s
a module 10 others require 0.81 s 1.75 s
the type module 32 others require 1.74 s 1.75 s

What per-module saves depends entirely on where the edit lands. Editing a leaf is a hundredfold; editing the module at the bottom of the dependency graph saves nothing at all, because everything above it has to be checked again anyway. Neither mode is slower than a cold check. whole-run keeps one entry per invocation rather than one per module, which is the reason to reach for it if the number of files in .htl/ becomes a problem before eviction lands.

An entry is used only when the module and everything it required still hash the same, every name it requires still resolves where it did, and the binary that wrote the entry is the one reading it. Content hashes throughout, no timestamps, so touching a file without editing it invalidates nothing and a fresh checkout does not either. Anything unexpected — a corrupt entry, an unreadable store, an htl upgrade — is a miss, which costs the check it would have skipped and never the wrong answer.

Only the names a module actually requires are watched. Adding a module nothing requires leaves every existing entry valid; adding one that could answer to a name something does require invalidates the modules asking for that name, whether or not the checker would still have picked the old file. Writing a new module is a normal thing to do while working, and it costs a check of that module rather than of the project.

The store is bounded, at four entries per module or 256, whichever is larger. A run that finds it over the bound drops what it did not itself use: entries whose files are gone go first, then the oldest until it fits. A dropped entry is a miss on the next run and nothing worse. Eviction is where mtimes are allowed, because being wrong there costs a check rather than a wrong answer; invalidation still refuses them.

Flags are part of the key when they change what a module reports and not when they only change the verdict: --lint gets its own entries, --strict reuses them and differs in the exit code alone.

htl test shares the store, for the half of its work that does not depend on the outcome: checking a test file and generating its Lua. The run is never cached — a test has to run to say whether it passes, and it does, every time. The summary says how many files had their checking reused (27 checked from cache), and --no-cache opts out as it does for htl check.

The modules a test requires are stored too, and put in front of the module searcher before the file runs, so requiring one does not check and generate it mid-execution. Measured on a 27-file suite: 4.65 s without any of this, 5.03 s on the first run (which stores what it generated) and 3.03 s on every run after.

htl cache status says what the store holds — entries by kind, total size, how recently they were used, and with --entries the files each one covers. htl cache clear empties it. Both find the store beside htl.toml, which is not necessarily where you are standing: htl check src run from anywhere in a repo writes to the project root.

--explain-cache (or HTL_CACHE_DEBUG=1) prints why each lookup missed and one line at the end with what the run did: hits, misses, entries written, entries evicted.

Embedding in Rust

use htl::{Htl, TealRecord, host_module, include_tl, include_tl_bytes};

#[derive(TealRecord, Clone)]           // Teal record <-> plain table (IntoLua / FromLua)
pub struct Point { pub x: f64, pub y: f64 }

pub struct Host { started: std::time::Instant }

#[host_module(name = "host", dts = "scripts/host.d.tl", records = [Point])]
impl Host {
    pub fn uptime_ms(&self) -> u64 { self.started.elapsed().as_millis() as u64 }
    pub fn scale(&self, p: Point, k: f64) -> Point { Point { x: p.x * k, y: p.y * k } }
    pub fn greet(name: &str) -> String { format!("hello, {name}") }            // static
    pub fn parse(s: &str) -> Result<i64, std::num::ParseIntError> { s.parse() } // Err -> Lua error
}

const MAIN: &str = include_tl!("scripts/main.tl");              // checked at cargo build
const UTIL: &[u8] = include_tl_bytes!("scripts/util.tl");       // same, as stripped bytecode

fn main() -> anyhow::Result<()> {
    let h = Htl::new()?;
    Host { started: std::time::Instant::now() }.htl_preload(&h)?;
    h.preload_bytes("util", UTIL)?;
    h.exec(MAIN, "=main.tl", &[])?;
    Ok(())
}

exec passes its arguments to the script as ... and nothing else. A script that reads arg[1], as htl run lets it, needs h.set_arg("main.tl", &args)? before exec: that fills the arg table the way the lua CLI and htl run do, so the same main.tl runs unchanged both ways (htl new --embed writes both calls).

#[derive(TealRecord)] is checked in one direction at build time and one at runtime: the .d.tl it writes is what the Teal side is compiled against, while a table coming back the other way is compared field by field as it converts. A table that does not fit says which record, which field, what the record declared and what arrived:

Outcome.cause: expected string, got nil
Outcome.depth: expected integer, got string
Recording.outcome.cause: expected string, got nil

A Teal record literal may leave fields out and htl check is right to pass it, so this message is the whole signal for that direction; require_fields in htl.toml is the check-time counterpart when a module's table is meant to be complete.

Result<T, E> returns raise a Lua error on Err by default. With #[host_module(name = "store", errors = "return")] they come back Lua-style instead: Ok(v) -> v, nil, Ok(()) -> true, nil, Err(e) -> nil, tostring(e), and the .d.tl says function(...): T, string (boolean, string for unit), so local ok, err = store:write(name, text) needs no pcall.

#[host_module] turns the plain impl into a mlua::UserData impl and writes scripts/host.d.tl when it expands, so scripts/main.tl sees host:scale(p: Point, k: number): Point and host.Point. Change a Rust signature and the next cargo build fails inside the .tl that relied on it. &str, &[T] and &Record parameters are accepted (&mut is not); nested records come from structs in the same source file, records from other modules via uses = [Name] + their own .d.tl.

async fn (feature async)

A method may be async, in the same impl as the sync ones and with no annotation saying which the block contains. It is registered through mlua's async variant, and its Teal declaration is the one the same signature produces without async — a function that yields internally and hands back the same values is an ordinary call from Lua, and Teal has no way to say otherwise.

#[host_module(name = "api")]
impl Api {
    pub fn seen(&self) -> u32 { self.calls }
    pub async fn fetch(&self, path: String) -> String { /* … */ }
}

Three things follow from mlua, not from htl:

  • The executor is yours. mlua yields to whatever is polling and provides nothing of its own, so an async method runs under call_async (which creates the coroutine for you) or an AsyncThread you drive. Called from a plain load(..).eval() there is nothing to suspend, and Lua raises rather than blocking.
  • The receiver is borrowed across every await. add_async_method hands over a UserDataRef<T> that the future holds until it resolves, so nothing else may take the value exclusively meanwhile. Prefer &self over &mut self.
  • The future must be 'static, and Send as well when mlua's send feature is on.

The feature is off by default: it turns on mlua's async, and a host with no async method should be built as it was without it.

Runtime resolution through mlua-pkg:

let mut reg = mlua_pkg::Registry::new();
reg.add(NativeResolver::new().add("host", |lua| { /* Rust table */ }));
reg.add(htl::pkg::TealResolver::new("scripts")?);     // .tl / init.tl -> check + gen; .d.tl -> type-only table
reg.add(mlua_pkg::resolvers::FsResolver::new("scripts")?);
reg.install(h.lua())?;
// or, with an mlua-pkg.toml: htl::pkg::Project::find(dir)?.registry()

A .tl that fails its type check is Some(Err) in mlua-pkg's terms: it never falls through to a later resolver. Native modules must be registered before the Teal resolver and described by a .d.tl for the checker.

Teal resolves every require("literal") at check time, and htl keeps it that way. When a module exists only at run time (the user's Tasks.tl that a long-built host loads), the same two shapes that TypeScript, Kotlin scripting and Gradle use apply:

  • Declare it (declare module / .d.ts in TS terms): ship Tasks.d.tl in the host's tree with the contract (local tsk = require("tsk") local Tasks: tsk.Tasks return Tasks). The build checks the host's scripts against the declaration; at run time a TealResolver rooted at the user's project serves the real file.
  • Hand the user a typed constructor (defineConfig / satisfies UserConfig in TS terms): the SDK exports define: function(t: tsk.Tasks): tsk.Tasks and the user writes return tsk.define({ ... }). Field-level errors with line numbers, no annotation on the user's side, and expect_type becomes a belt-and-braces check.

A dynamic require(name_in_a_variable) typed as any is the escape hatch, like GDScript's load() or a shorthand declare module "x"; use it only when the module name itself is unknown until run time.

Errors that come out of running Lua (a host function's Err, a Lua error(...)) carry mlua's stack traceback:; htl::user_message(&err) returns the innermost cause alone, which is what htl run / htl test print.

For mod / plugin directories, TealResolver::new("mods")?.expect_type("defs.Mod") holds every served module to a record type: a mod that returns the wrong shape is rejected at require time even if it never annotates its own return value. It rejects fields of the wrong type; on its own it does not reject missing fields (every Teal record field is nilable). Chain .require_fields(["name", "monsters"]) to name the fields that must be present: the module is rejected at require naming the nil ones, and a field added to the record later stays optional until it is added to the list, so the type can grow without breaking the modules already written against it. .require_all_fields() takes every declared field, for types that are settled.

Lints (htl check, include_tl!)

rule default catches
nil-index on t[k].x, t[k]:m(), t[k](), t[k][j] — Teal types a map/array lookup as V, not V | nil
struct-fields on a table built for a record marked ---@struct that leaves out a field the record declares and ---@optional does not exempt. Silent until a record carries the marker (see below)
enum-exhaustive on if e == "a" ... elseif e == "b" ... end over an enum with a value left unhandled and no else; enums nested in records and enums from required modules count
union-exhaustive on if x is A ... elseif x is B ... end over a union with a variant never tested and no else. The variants come from the checker, so a chain that predates a variant is reported once the union gains it (see "Unions of records")
shadow-local on a local / loop var / parameter reusing an enclosing local's name; when that outer local is a required module the message says which module and where it was required
no-global on global declarations
no-any off explicit any annotations and as any casts
explicit-number off local n = 0 (inferred integer) that is later assigned a number expression (n = n * 1.5, n = a / b): names the declaration and the assignment; write local n: number = 0. Plain integer counters are not reported
class-record off a record declaring metamethods (metamethod __index: Actor = a class): its metatable is attached by setmetatable at run time and is not part of the value, so serialization and the Rust boundary drop it; keep such records out of saved data and host signatures
require-cycle on (project-level) a loop in the require graph of the files htl check <dir> just checked, e.g. a.tl -> b.tl -> a.tl. Teal types the back edge as an opaque circular require, so without this the symptom is "cannot index" somewhere else

Silence one occurrence with a trailing -- htl: allow(nil-index). include_tl! treats lints as errors (HTL_LINT=warn downgrades, HTL_LINTS=+no-any,-shadow-local configures).

Records built whole (---@struct)

Every Teal record field is nilable and Teal has ? for function parameters but not for record fields, so a record the program builds itself still reads as if any field might be absent. ---@struct says it does not:

local record MonsterDef   ---@struct
   id: string
   hp: integer
   inflicts: Status       ---@optional
   ---@optional
   home: BranchId
end

Every table built as a MonsterDef must then set id and hp; inflicts and home may be absent. Adding an unmarked field makes the construction sites that predate it report, which is the point — the default for a new field is mandatory, and ---@optional is the exception you write on purpose.

A misspelled field is the case where two rules each hold half the answer: the checker says unknown field colour about the key that exists, and this says color is missing. When the key the literal sets is a near miss for the one it wants, the message names it instead of repeating the standing advice, because "mark it ---@optional" is the wrong fix for a typo:

MonsterDef is built without color (the literal sets `colour`)

One edit counts as a near miss in any name, two once the name is at least eight characters long. An extra key that is nothing like the missing one is not offered.

The markers go where the record is declared, and the report lands where it is built, so an SDK can declare the shape its mods must fill in. Both marker forms work: trailing on the field's own line, or on the line above it. Every construction site counts — a bare literal, an element of an array or map of that record, a literal passed as a typed argument, and a function's return.

This is a lint, not a type. The file stays valid Teal and other tooling ignores the comment; use sites still see a nilable field. What it removes is the reason to guard, and the doubt about whether a field was ever set. Data arriving from outside the program — a mod's return value, a save file, a host — is a different question, and [[contract]] with require_fields is what checks that.

Project config (htl.toml)

htl check / htl test / htl fmt / include_tl! all read the nearest htl.toml above the file, so the CLI and the build agree. Flags and HTL_LINTS / HTL_LINT override it (htl new writes a commented one).

[lint]
enable  = ["class-record", "explicit-number"]
disable = ["shadow-local"]
strict  = true            # lints fail check/test and include_tl!; false makes the macro advisory

[fmt]
indent = 3

[check]
paths = ["mods", "~/.cache/tsk/sdk"]   # extra dirs require() resolves from while checking

[[contract]]              # where this project accepts modules written outside it
dir = "mods"              # relative to htl.toml; "sites/*" = every subdirectory of sites/
# module = "Site"         # optional: only this module name (in each dir) is held to it

[check] paths is for modules the host supplies at run time from somewhere the checker would not look (an SDK cache, a mods dir): the CLI, include_tl! and contract_resolvers all add them, plus the htl.toml dir, its src/ and its types/. types/ is the conventional home for .d.tl (the DefinitelyTyped shape: declarations the module's author did not ship), searched without any configuration; htl new creates it.

Three kinds arrive there. The ones written by hand; the ones a dependency published; and the ones for a library that published none of its own.

A package keeps its own declarations at types/ in its root, which is outside the entry directory require resolves through, so htl pkg copies them in. htl types add <library> is the other half: teal-types is where the Teal ecosystem collects declarations for libraries that ship none, as types/<library>/<module>.d.tl, and add takes one library's worth. The library's own directory is dropped and the path below it kept, since that path is the module name — socket/http.d.tl stays require("socket.http").

Both write a .src note beside each file: what published it, at which commit, and the path it had there. Nothing else records that. luasocket-tl-type is versioned 0.0.2-1 against a luasocket at 3.x, its rockspec declares no dependency on luasocket, and its source names no revision — so without the note, a declaration carries no evidence of what it was written against.

Copying rather than searching the installed deps is what makes them survive a fresh clone: .htl/ is gitignored and empty until someone installs, types/ is committed. A name types/ already has is reported and left alone (--force replaces it): two libraries publishing a module of the same name is a real situation, and there is no registry to arbitrate it with.

Source beats declaration: when both defs.tl and a defs.d.tl are reachable, the checker reads the .tl, wherever the two sit on the path (Teal's own order is .d.tl first). So a .d.tl a host writes out for external script authors never shadows the source it was made from inside the repo, and a check that runs before the host has rewritten it still sees the current types.

Between two declarations of one module there is no such rule, only position: the directories above are consulted in the order they are listed, and the first hit is the one read. duplicate-declaration reports it — a project that keeps a hand-written xlib.d.tl under types/ and also has one arriving from a [check] paths directory is told which is in effect and which is not, rather than being left to work out why a type is not what the file in front of it says.

Data from outside the program (---@contract)

htl.toml says where modules arrive; the record says what they must be. Marking the record is what makes the contract discoverable — a directory carries no evidence of which of a project's records is the one its modules must satisfy.

local record defs
   record Mod              ---@contract
      name: string         ---@required
      monsters: {Monster}  ---@required
      items: {Item}        ---@required
      factions: {Faction}
      npcs: {Npc}
   end
end
return defs

Every module directly under mods/ must return a value assignable to defs.Mod and set the three marked fields. factions and npcs are for the mods that want them, and that asymmetry is the point: a record cannot say which of its own fields are mandatory (every Teal record field is nilable and there is no ? for them), and holding modules to all of them would break every one written before a field was added. Marking the mandatory ones lets the type grow.

The default is the opposite of ---@struct's, and each marker says which regime its record is under: ---@struct is about a record the program builds itself, where a new field is mandatory unless marked ---@optional; ---@contract is about a value arriving from outside, where a new field is optional unless marked ---@required.

A bare ---@contract inherits the directory from htl.toml, which is what a project with one contract writes. ---@contract("plugins") names its own, ---@contract(module = "S") narrows a directory to one module name, and both can be given at once.

The module the contract type is declared in is what an outside author writes their modules against, so htl publishes it: types/defs.d.tl here, alongside the .d.tl a Rust host's #[host_module] writes, regenerated by htl dts and by check / run / test / build. ---@contract(dts = "sdk/defs.d.tl") sends it somewhere else. Commit the result, the same as the Rust-generated ones: it is what makes a fresh clone check before anything has been built.

What it writes is the declaring module with its bodies removed: each function the module exported becomes a field of the record it was on, keeping its parameter names and its doc comment, which is what a hand-written .d.tl says.

function defs.describe(m: Mod): string    -->    describe: function(m: Mod): string
   return m.name
end

A local function is not part of what the module declares and leaves nothing behind, a field the record already declares is left as the author wrote it, and a method keeps the self its definition left implicit. The published marker names its directory outright, since whoever reads the declaration does not have the htl.toml a bare ---@contract inherits from. A function on a table the module declares no record for is reported rather than dropped — a declaration missing a function is worse than one that was not written.

Two lints follow:

  • contract — a module under the directory whose return value is not assignable to the record, or whose returned table literal leaves a ---@required field out, is reported at htl check time instead of at the first require. The literal is found through return { … }, return define({ … }), return { … } as T, and local m: T = { … } … m.f = … return m. A marker that cannot be turned into a contract — one naming no directory in a project whose htl.toml declares none or several, two markers claiming one directory, a marker on the record a module returns rather than on one inside it — is reported here too.
  • contract-unenforced — a contract is only a guarantee if the host enforces it. When a Cargo package is found, htl check scans its Rust sources for contract_resolvers( and otherwise tells you to add it, or to say where it is enforced with enforced_by.

Hosts build their resolvers from the same markers, so the two cannot drift:

let (path, cfg) = htl::config::HtlConfig::find(Path::new("."))?.expect("htl.toml");
let mut reg = mlua_pkg::Registry::new();
for r in htl::pkg::contract_resolvers(&htl::parent_dir(&path), &cfg)? {
    reg.add(r); // TealResolver for <root>/mods, expecting the record marked
                // ---@contract for that directory and its ---@required fields
}

That one call is what contract-unenforced looks for. A resolver assembled by hand from TealResolver::new(…).expect_type(…).require_fields([…]) still works, but it restates what the record already says, which is the drift the marker exists to remove.

Enforcement the scan cannot see — a Lua-side validator that checks the table before the host uses it, a resolver in a sibling crate, generated code, or one built by hand on purpose — is named instead:

[[contract]]
dir = "mods"
enforced_by = "mods/_validate.lua"   # relative to htl.toml; ~ and absolute paths work

That contract is then not held to the scan, and the others in the project still are. It takes a path rather than a true because the file has to exist: a name that points at nothing is reported under the same rule, whether or not the call was found elsewhere, so the key stays a claim htl check can hold to something rather than a per-contract off switch.

Patched dependencies (htl pkg patch)

A dependency needs one line changed. htl pkg patch mathx copies its package root — the whole package, so its types/ comes with it — out of the pinned revision and into patches/mathx/, writes patch_dir = "patches/mathx" onto that dependency in mlua-pkg.toml, and records the commit it was taken from as patch_base in the lockfile.

  patched patches/mathx (mathx at 3f2a9c1)

From there the directory is the project's code: edited, diffed, reviewed and committed with git like anything else in the tree. There is no patch file and nothing is applied — htl pkg install leaves the directory alone and resolves the dependency from it. This is the shape of Cargo's [patch] with a path source, and of Go's replace pointing at a directory in the module tree. Removing patch_dir and the directory returns the dependency to its fetched form at the next install.

What is checked, and what is not. The copy is committed, project-owned code whose errors are the project's to fix, so htl check walks it and names the dependency each directory stands in for. htl fmt and htl test do not touch it: formatting it would turn every file into a diff against its base and hide the change inside it, and its *_test.tl are the dependency's suite rather than the project's. .htl/modules is not descended into at all, patched or otherwise; its modules are checked through the require that reaches them and their errors reported against the requirer, never offered to htl fix (a fix there would go at the next install — patching is how a dependency is edited). The criterion for walking is who writes the directory — one that install regenerates (target_dir) is skipped, one that the project edits is checked.

Upgrading. A patch is bound to the revision it was taken from. When the pin moves — the dependency was upgraded — install fetches the new revision and resolves from it, the copy is left alone, and every install says so until the patch is refreshed or removed:

  patch   patches/mathx is not in use (taken from 3f2a9c1, mathx is now at 8b07e44)
          carry the change forward: commit it, then `htl pkg patch mathx`
          drop it: remove patch_dir from mlua-pkg.toml and delete patches/mathx

Install does not fail over it; the project builds against the new upstream. htl pkg patch on an already patched dependency refreshes the copy from the revision the pin now resolves to and records that as the new base. The copy is overwritten rather than merged, so carrying the project's own change forward onto it is a merge git performs — which is why a directory with uncommitted changes is refused, naming them, and why --force (which discards them) is a flag rather than the default. Outside a repository the question cannot be asked at all, and that is said rather than guessed at.

Tests

local t = require("htl.test")            -- typed via test.d.tl
t.describe("util.add", function()
   t.it("adds", function()
      t.expect(util.add({x=1,y=2}, {x=10,y=20})):to_equal({x=11,y=22})
   end)
end)

expect(x) is generic, so t.expect(1 + 1):to_equal("2") is a type error and the file is refused before it runs.

The split follows Go / Rust rather than Jest: htl invests in the runner and keeps the assertion surface small enough to read in one screen. Matchers: to_equal, to_not_equal, to_be_truthy / to_be_falsy, to_be_nil / to_not_be_nil, to_be_close, to_be_greater_than / to_be_less_than / to_be_at_least / to_be_at_most, to_contain / to_not_contain (substring or array element), to_match / to_not_match (Lua pattern), to_have_length, to_error. A function returning two values is asserted with t.expect_all(f()):to_equal(false, "no door") (t.expect(f()) is a 2-argument call and a type error; the message says so).

Snapshots: t.expect(session.frame(s)):to_match_snapshot("first floor") compares the value with tests/__snapshots__/<test file>/<name>.snap. The first run writes the file (and says so); later runs fail with a -expected +actual line diff when the value changed; htl test --update rewrites the differing ones. A string is stored as is, an array of strings as its lines (a rendered screen), anything else in a sorted, one-entry-per-line form, so the files read well in a review. A name used twice in one file is an error.

Coverage: htl test --coverage prints, per .tl module the tests' checks depended on, how many of its statements ran (executed/all %), and a total; a module no test reached shows 0/n. Under a module it names the functions nothing entered, since the percentage says how much was missed and not what:

coverage: src/combat.tl      124/181   68.4%
          never ran: resolve_counter (61), flee_path (130)

--coverage-lines adds the unexecuted line ranges under those. Statements are counted from the .tl syntax tree and matched against Lua's line hook (Teal keeps line numbers when it generates Lua), so the numbers are .tl lines. A function counts as entered when a line strictly between its function and its end ran: defining a function runs both of those lines, so neither says anything about calls. A function with nothing in between — written on one line, or with an empty body — is not reported. The hook slows the run, and code that runs inside a coroutine the program creates is not seen.

--lcov coverage.info writes the same run as an lcov tracefile, which is what Codecov, Coveralls, GitLab, genhtml and editor gutters read; it implies --coverage, and the table and --format json are unchanged. One record per module the table lists: FN / FNDA from the functions above (1 when the body was entered, 0 when not), DA per line a statement starts on, with a count of 1 or 0 — the hook records whether a line ran, not how often, and consumers treat any non-zero as covered. Two statements starting on one line share the entry, so LF / LH differ from the table's total / executed by exactly those lines. There is no branch data and no BRDA. SF is relative to the project root (the htl.toml directory) rather than to where the command ran, so the file resolves against the repository wherever CI stood; a module outside the root is written absolute.

Randomness: the runner seeds each file before it runs, prints the seed of every run, and takes it back with --seed, so a test that draws is one whose failure can be looked at again:

htl test: seed 8014255196 (repeat with --seed 8014255196)

t.rng() is that stream, shaped like math.random (rng(), rng(m), rng(m, n)); math.random is the same stream, so a test already using it repeats too. Each file's seed is derived from the run's seed and the file's own path rather than drawn from one shared stream, so what a file draws does not depend on which other files ran or in what order — running it alone, or with --filter, reproduces what it did in the full run. A test that calls math.randomseed itself takes over from there; the runner does not seed again.

Runner: htl test [paths] [--filter substr] [--fail-fast] [-v | -q] [--slow MS] [--update] [--seed N] [--coverage [--coverage-lines]] [--lcov FILE]. Each file runs in a fresh state; -v prints every test with its time, -q only failures (with their details), errors and the summary line, --slow 50 the tests over 50 ms, --fail-fast stops at the first failure. The run has one checker (htl::testing::TestSession) and one fresh program state per file: globals, package.loaded and module state never cross files, while a module is type-checked and generated once and served to every file whose search path resolves that name to the same file (Htl::with_checker is the same split for hosts that run many programs). HTL_PROFILE=1 prints per-phase and per-file timings to stderr. Any library exposing run(filter, opts) -> {passed, failed, failures, tests?, snapshots_written?, snapshots_updated?} (and optionally configure({snapshot_dir, update, mkdir})) plugs in via --lib (bring its .d.tl); files that use no such library pass if they run to completion.

Fixing (htl fix)

Some diagnostics carry a mechanical fix; htl check marks them (fixable: htl fix) and --format json carries the edits. htl fix [paths] applies them:

  • Every fix has an applicability: safe (what the program does at run time is unchanged), unsafe (it may change; applied only with --unsafe), suggest (shown, never applied). Today: a forward reference gets its declaration inserted into the record (safe); explicit-number gets : number (safe); no-global becomes local (unsafe). htl.toml [fix] unsafe = ["no-global"] promotes a rule, disable = [..] turns its fix off; --rule a,b limits a run.
  • The working tree is the undo. A file git reports as modified or staged is refused (--allow-dirty), and so is a file outside a repository (--allow-no-vcs). --dry-run reports without writing; --diff prints a unified diff per file instead.
  • A file with a syntax error is never touched. Type errors elsewhere do not block (a fix is often what removes one); after each pass the file is re-checked and put back if it has more errors than before. Edits that overlap an applied one wait for the next pass; passes are capped at 4; two passes producing the same edits are reported as fixes undoing each other.
  • Everything applied is listed (fixed: file:line: rule (safe)), as is everything skipped and why. Exit code as htl check (remaining errors → 1); --exit-non-zero-on-fix also fails when a file changed, for CI.

Machine-readable output

htl check --format json and htl test --format json print one JSON document on stdout and nothing on stderr (the text form is stderr-only, so the two never mix). The exit code is the same as in text mode. Field names are stable; fields may be added, not renamed.

  • check: { files, diagnostics: [{ severity: "error"|"warning"|"lint", file, line, col, rule?, message, required_by?, origin? }], summary: { errors, warnings, lints, strict, ok } }. rule is the lint rule (nil-index, contract, ...), split out of the message. An error in a module the check reached through require has file set to that module and required_by to the file that required it; origin is "dependency" (installed under .htl/modules, or a vendored copy) or "external" (a [check] paths or contract directory), and absent for a file of the project's own.
  • test: { files: [{ path, ok, diagnostics, error?, file_level, passed, failed, failures, tests: [{ name, ok, ms }], duration_ms, snapshots_written, snapshots_updated }], summary: { files, files_run, passed, failed, files_with_errors, duration_ms, ok, seed }, coverage?: { modules: [{ path, executed, total, unexecuted: [[first, last]], never_ran?: [{ name, line }] }], executed, total } } (coverage with --coverage; never_ran is absent when every function of the module ran).

GitHub Actions annotations from a check, for instance:

htl check . --format json | jq -r '.diagnostics[] |
  "::\(if .severity == "error" then "error" else "warning" end) file=\(.file),line=\(.line),col=\(.col)::\(.message)"'

Bundles (htl build)

htl build src/main.tl -o app.hb follows require("<literal>") from the entry and links everything it reaches into one file: .tl modules type-checked and generated, plain .lua modules (a dependency's own sources) as they are. A require that resolves only to a .d.tl is recorded as host-provided (a Rust #[host_module], a preload); any other unresolved require is a build error, so "module not found" happens here and not on the first require at the user's machine. htl run app.hb runs it; a host does Htl::run_bundle(&Bundle::decode(bytes)?, &args) after registering its modules, and is refused up front, naming them, if one is missing.

  • Payload is stripped Lua 5.4 bytecode by default. --debug keeps line numbers and local names (tracebacks with lines; module names survive stripping since the loader supplies them). --source stores generated Lua instead: larger and readable, and bound to no Lua build.
  • Portability. A bytecode bundle runs on any host whose Lua chunk header matches the one it was compiled by: version, bytecode format, the sizes of instruction / integer / number, and endianness. Nothing about the CPU or the OS is in a Lua chunk, and the Lua htl vendors has a 4-byte instruction, 8-byte integer and 8-byte double on every 64-bit little-endian platform, so a bundle built on an arm64 Mac loads on x86_64 Linux and cross-building between mainstream desktop and server targets needs nothing. --source is for the cases the header refuses: a big-endian target, a host whose Lua was built with a non-default LUA_INT_TYPE / LUA_FLOAT_TYPE, and a bundle that has to outlive a Lua upgrade (mlua pins the Lua htl vendors, and a new one may change the format). install_bundle checks the header before the first require and refuses on mismatch, naming both sides: compiled for Lua 5.4, format 0, 4/8/8, little-endian by htl 0.1.19, but this host runs ... on htl 0.2.0. The htl version is advisory (a bundle from an older htl whose header agrees still loads); it is in the message because the header alone cannot say why two 5.4 builds disagree.
  • htl bundle info app.hb prints what the file records — format version, the htl that built it, payload kind, the Lua the bytecode is for in the same words as the mismatch message, entry, modules, host-provided names — without creating a Lua state. That is what a build step checks in and a bug report pastes; --format json for the same. A --source bundle says its Lua is any; a format 1 bundle (HTLB\x01, before the fingerprint) says it was not recorded.
  • A dynamic require(expr) cannot be followed: list its targets under [build] extra in htl.toml (or --extra). Modules the host provides without a .d.tl go under [build] host (or --host).
  • Bundled modules are installed as package.preload entries, the same place a host puts its own (a name the host preloaded first is left alone: the host wins). So everything that defers to preload, a .d.tl stepping aside for the implementation or an mlua-pkg resolver over a mods dir, sees bundled modules too, and files on disk do not override the bundle.
  • htl build <dir> (the older form) still bundles every .tl under a directory.

From Rust, include_bundle! does the same at cargo build and keeps the guarantee include_tl! gives a single file: every linked .tl / .lua / .d.tl is tracked, so an edit rebuilds, and a Teal type error anywhere in the closure fails the build.

const BUNDLE: &[u8] = htl::include_bundle!("src/main.tl", host = ["host"], extra = ["modkit"]);
// payload = "source" for a target whose Lua header differs (big-endian, non-default
// number types; see Portability above); debug = true keeps line numbers.
// [build] extra / host in htl.toml are merged in.
Host { .. }.htl_preload(&h)?;
h.run_bundle(&htl::bundle::Bundle::decode(BUNDLE)?, &args)?;

Doing the same from a build.rs with htl::link::link works too: take the bundle through Linked::bundle() / into_bundle() (an Err lists every type error; link itself returns Ok so the whole list can be shown, and never hands out a bundle with a module missing), and emit cargo:rerun-if-changed=<file> for each of Linked::inputs(). Name files, not the directory: cargo compares the mtime of the path it is given, and editing a file inside a directory does not change the directory's.

Layout of a project (htl new)

<name>/
├── mlua-pkg.toml          [package] entry = "src/<mod>"  → consumers require("<name>")
├── htl.toml               [lint] / [fmt] / [[contract]] shared by the CLI and include_tl!
├── src/<mod>/init.tl      the module (require("<mod>") from src/ and tests/)
├── types/                 .d.tl the project consumes (hand-written) and publishes
│                          (a ---@contract type), searched by default
├── patches/<dep>/         a dependency taken into the tree (htl pkg patch), committed;
│                          checked, not formatted, its tests not run
├── src/main.tl            entry script
└── tests/<mod>_test.tl

mlua-pkg's entry is a directory, so a consumer's require("<name>") looks for <name>/init.tl. A flat package can instead ship <name>/<name>.tl (e.g. entry = "src" with src/<name>.tl); htl resolves that form in the checker and in TealResolver.

Unions of records (where)

Teal refuses a union of two record types on its own:

cannot discriminate a union between multiple table types: A | B

The refusal is about run time, not syntax: is narrows with a type() check, and two records are both table. A record can supply its own discriminator with a where clause, and then the union type-checks and is narrows it:

local record Monster
   where self.kind == "monster"
   kind: string
   hp: integer
end

local record Item
   where self.kind == "item"
   kind: string
   weight: number
end

local function describe(e: Monster | Item): string
   if e is Monster then
      return "hp " .. tostring(e.hp)      -- e.weight here is an error
   else
      return "weight " .. tostring(e.weight)
   end
end

where takes an expression that uses self once; comparing an enum-typed tag field works the same way and is the usual shape. Inside a narrowed branch the other variant's fields are not in scope — reaching for one is invalid key 'weight' in record 'e' of type Monster — and a partially narrowed value keeps its remaining variants, so after is A over A | B | C the value is B | C and a field only B has is still an error.

That "once" is the cost of the form, and it decides where the form belongs. One record cannot answer to two tag values:

cannot use argument 'self' multiple times in macroexp

So a type with seven tag values needs seven records, and it is worth writing them only when the variants carry different data. Where several tags carry the same data, a union buys nothing an enum field on one record does not already give: the branches are guarded by enum-exhaustive either way, and the declarations are the only thing that grew.

A worked example from a project that decided against one. Its Effect has five fields and seven tag values, but only four payload shapes among them — power, power + damage, status, and nothing at all. As a union that is seven records, four of them structurally identical, around thirty lines of declaration, to gain field safety at the one place it is read. It stayed an enum plus a record, and that was the right call.

The question to ask is not "does this have a tag" — plenty of records do — but "do the variants hold different things". When they do, the union pays for itself at every use site. When they do not, the tag was already saying it.

A variant nobody handled is not a type error — an is chain that covers A and B and falls through compiles, and goes on compiling when C joins the union — so the union-exhaustive lint reports it. It reads the union's members from the checker rather than from the tests, and stays quiet for a chain with an else, for a single is (that is a guard, not a dispatch), and where every branch returns and code follows, which is the else written differently. Those are the same exemptions enum-exhaustive makes.

This is a Teal feature, not an htl one; it is documented here because the error above is what a reader meets first, and it reads like a dead end rather than a pointer to where.

Pitfalls the checker now names

  • Case-insensitive filesystems (macOS, Windows): require("site") from a file called Site.tl resolves to that very file. Teal reports it as "no type information for required module"; htl appends that the module resolved to the requiring file itself and that one of the names has to change.
  • Numeric inference: local n = 0 is integer, 0.0 is number; opt into the explicit-number lint to be told where an annotation is missing.
  • Forward references: function world.tick calling world.observe that is defined further down is "invalid key 'observe' in record 'world'", because Teal adds a record's fields in source order. htl names the later definition and hands over the line to paste into the record (observe: function(w: World, what: string)), which also makes the record the module's declared API; moving the definition up is the other fix.
  • A union of two records: "cannot discriminate a union between multiple table types" reads like a limit on the type system, and it is a limit on is, which each record can lift for itself with a where clause (see above).
  • Multi-value call in last position: t.expect(can_cast(x)) with can_cast returning boolean, string is a 2-argument call, and Teal reports "wrong number of arguments" at expect. htl names the expanding call and the two fixes (bind first, or parenthesize to keep the first value).

Releasing and using a local checkout

docs/releasing.md: the publish order, the crates.io per-crate 24-hour version limit and what to do when it hits, and how a consumer runs against an unpublished htl with [patch.crates-io] (all three crates, plus one cargo update -p htl -p htl-core -p htl-macros, without which cargo keeps the locked version and warns that the patch was not used).

What is deliberately not here

  • No Teal fork: tl.lua is vendored verbatim (0.24.8, MIT) and swapped as a file.
  • No token-level formatting: htl fmt recomputes indentation and whitespace only.
  • No Luau: PUC Lua 5.4 / LuaJIT via mlua features. Bytecode bundles are bound to the Lua chunk header of the htl that built them, which every 64-bit little-endian host shares (see Bundles, Portability); there is no dual bytecode-plus-source payload, since shipping the source is what the bytecode form exists to avoid.
  • .d.tl files are written syntactically, from Rust source (htl dts, and the macros at expansion time write the same text) and from Teal (the module a ---@contract type is declared in). There is no reflection on types either way: a Rust field of type Foo is declared as Foo and it is on you that a Teal Foo exists, and a Teal signature is carried across as it was written.

License

MIT OR Apache-2.0. Teal (crates/htl-core/vendor/tl.lua) is MIT, see crates/htl-core/vendor/LICENSE.teal.