termproof
Evidence-first verification for TUI and terminal applications — the library half of TermProof.
Maturity: this port is in progress and is not at parity with the Python implementation. The Python implementation at
md-mt/termproofis the shipped product and the behavioural oracle for TermProof; there is no parity gate for this port. Read status and parity before depending on this crate.
Layout
This crate was merged from three — termproof-core, termproof-terminal and
termproof-evidence — before any of them was published, so the shape below is
the only one that has ever existed on crates.io.
- The crate root is the recipe model, config, schema, validation, steps,
assertions, planning, orchestration and execution. It is flat rather than
under a
coremodule: it is the crate's primary surface, and a module namedcoreshadows thecorecrate for every path in its scope. terminalis the session layer — PTY, tmux and process sessions, plain and attributed screen state, asciicast recording.evidenceis the evidence pipeline — screenshot and video rendering, Markdown reports, visual baselines, diff and upload.junitis the JUnit XML writer. It lived intermproof-evidencetoo, andevidence::report::generate_junitstill resolves, but it reads aRunResultand renders nothing, so it is its own module and its own feature.
Features
The first four are on by default, so termproof = "0.3" is the whole crate as
it has always been published. clap is the exception and is opt-in.
| Feature | Default | Enables | Costs |
|---|---|---|---|
evidence |
on | the evidence module |
image, avt |
junit |
on | the junit module, and generate_junit in evidence |
quick-junit |
json-schema |
on | validation, pyschema, and the json_schema built-in assertion. Implies schema |
jsonschema |
schema |
on | the schema module, and JsonSchema on Recipe, VerifierConfig and the types they contain |
schemars |
clap |
off | run_config::cli — a clap::Command of the standard RunConfig flags, and the parse back into one |
clap, builder-only (no clap_derive) |
The default-on four and the default-off one are default-on and default-off for
the same reason. Each of the four subtracts from a shape that was already
published, so on by default is what keeps turning one off from being a break.
clap adds a shape that never was, so off by default is what keeps it from
being a change: a consumer that fills in a RunConfig from a file, or from its
own parser, does not compile clap for a layer it will not call.
Turning the four off takes the crate from 180 transitive dependencies to 66:
= { = "0.3", = false }
That build still has the whole session layer and the other seven built-in assertions; what it loses is the evidence pipeline, JUnit output, JSON Schema validation and schema generation.
junit is the only feature that neither implies another nor is implied by one.
It was proposed as junit = ["evidence", ...], but generate_junit takes a
&[RunResult] and reads nothing else, so making it depend on evidence would
have charged a JUnit-only consumer image and avt for nothing. Either half
can be had without the other, and each costs only its own crates:
# stills, video and Markdown, no JUnit — six crates lighter than 0.3.1's
# `evidence`: quick-junit, quick-xml, newtype-uuid, uuid, strip-ansi-escapes, vte
= { = "0.3", = false, = ["evidence"] }
# JUnit, no renderers — those same six crates and nothing else, fifteen lighter
# than asking for both
= { = "0.3", = false, = ["junit"] }
schema is the small one — six crates, against json-schema's 87 — and it is
not off by default and not off for size. It exists because schemars is the
only dependency that reaches this crate's public API: the derives put
JsonSchema on published types, so a consumer already on a different schemars
major cannot deduplicate by pinning the way it can for everything else here, and
has to carry both. Turning schema off means this crate does not name
schemars at all, and that second copy is gone.
There is no terminal feature. The crate root is built on terminal and
nothing in terminal depends on the root, so a build without it would not be
this crate; it would also save nothing, since every build drives a terminal.
Dependency floors
Every version requirement here is a floor — the lowest version this code compiles and passes the differential harnesses against — and CI runs the suite at each one, so a floor that stops being true fails the build. If you already pin one of these elsewhere, two of them will not move, and the reason is not guessable from the import lists:
portable-pty0.9, not 0.8. Every nameterminal::ptyimports exists in 0.8 too. What does not isExitStatus::signal(), added in 0.9.0 and read byPtySession::exit_signal; at 0.8.1 the signal name is a private field, and that is the only compile error at 0.8. It is a choice, not an impossibility — 0.8 still shows the name throughDisplay, and scraping it back out returns the same value — but aDisplayformat carries no semver guarantee, so that version would answerNoneafter a reword with nothing to catch it. The floor buys one public accessor; nothing else in the crate reads it, and it is not on theSessiontrait.unicode-width0.2, not 0.1.vt1000.16 requires^0.2.1, and this crate's own width calls have to consult the same table as thevt100grid they describe or the two disagree about which column a wide glyph occupies — which moves glyphs in rendered SVGs and changesAttributedScreen::render_fingerprint, so evidence dedup stops recognising two identical screens as identical. Forcing 0.1 splits the graph in two and fails three tests interminal::attributedthat exist to catch exactly that. Widening the requirement to>=0.1, <0.3would resolve to 0.2 anyway, so it would buy nothing and only make the floor untestable.
Everything else is a plain floor with no such constraint; Cargo.toml carries
a comment on each one that sits above the oldest workable version.
What it provides
Root — recipes, steps, assertions, orchestration
Recipe,Step,Assertion,VerifierConfig— the recipe model, loaded from JSON or YAML, with a Draft 2020-12 schema and structured validation.steps— the seven built-in step actions and their dispatch.assertions— the eight built-in assertions.planner/runner/execution— recipe × renderer planning and execution against aterminalsession.store/cache— canonical artifact storage with a path-traversal guard, and a content-addressed run cache.pyregex/pyrepr/pypath/pyschema— the compatibility shims that keep this port's behaviour close to the Python oracle's.
terminal — sessions and screen
PtySession— a child process on a real pseudo-terminal viaportable-pty, implementing theSessioninterface the rest of TermProof runs against.TerminalScreen— avt100cell grid that interprets escape sequences rather than stripping them.CastRecorder/replay_cast— asciicast v2 recording and replay.wait_for_idle/IdleTracker— output-quiescence detection.SessionBackendimplementations:PtySessionBackend(the default),PluginSessionBackend, andInMemorySessionfor tests.
evidence — screenshots, video, reports
EvidenceCollector— an ordered step model for a caller driving its own run.captureandcapture_failurepull from aScreenSource;capture_text(label, screen, kind)records a screen the caller already holds, and takes theCaptureKindpositionally. This is the one collector signature the two implementations do not share: Python's iscapture_text(label, screen, kind=CaptureKind.CHECKPOINT), so the Rust call spells outCaptureKind::Checkpointwhere the Python one may omit it. The meaning, the ordering and the resulting manifest are identical.EvidenceCollector::record_session— the five steps around a whole-session video: save the live session's cast through the closure you pass, append the captured checkpoints to it as held trailing frames, convert it through the publisher'swith_video_converter, upload it, and attach aRecordingthatpublishwrites into the manifest. It returns nothing, because no step may fail the run: each failure lands on theRecording'serrorprefixed with the step that produced it —save cast,append checkpoint frames,convertorupload. A step runs only when the step before it produced what it works on, so a failed save skips everything after it and a failed conversion is never uploaded; a failed append is the one non-fatal case. A step is not believed either: a closure that writes no cast, a converter that returns a path it did not write, and an uploader that returns an empty string are each recorded as that step failing, rather than reaching the manifest as a recording that claims a video nobody can open.render_svg/render_by_extension— plain screen text to an SVG, in default colours.ScreenshotRenderer— an attributed grid to a PNG, with the colours the terminal actually showed. It andrender_svgdraw through the samescreen_svg, so an SVG still looks the same whichever produced it; pick on what you have, not on how you want it to look.render_png— plain screen text to a PNG with no external tool. It bundles no font, so it paints a block per occupied cell rather than glyphs: it shares the canvas, grid and palette withrender_svgand nothing else. A PNG that looks like the SVG is a rasterised one, which isScreenshotRenderer.evidence::render's module docs have the table.generate_markdown— a human report for a run. Its machine counterpart isjunit::generate_junit, behind thejunitfeature.apply_visual_diff— compare a screenshot against a stored baseline, or refresh the baseline.render_mp4— video via externalaggandffmpegbinaries, resolved at run time and failing with a named diagnostic when absent.
Library surface no TermProof command uses yet
Fourteen tested APIs with no caller in the CLI. Useful if you are building on
the library; not evidence that a termproof run does any of it.
docs/status-and-parity.md
lists the same fourteen.
parity— compares two runs and reports where they disagree.before_after— reports which outcomes flipped between two runs.selection— maps a changeset onto the recipes it affects, viaci_paths.run_config— a whole run described by one file, and under theclapfeaturerun_config::cli, the command line that fills one in.vocabulary— a configurable failure detector.build_info— provenance for the binary under test, so a result can be traced back to an exact artifact.terminal::attributed— a per-cell screen carrying foreground, background, bold, dim, italic, underline, strikethrough, reverse and display width, with an SVG renderer. A run writes no image at all today —raw_output.txt,screen.txtand a cast if one was recorded — so nothing in the CLI reaches this or the text path.terminal::tmux— aSessionthat runs the program in a tmux pane and reads the grid back withcapture-pane. A disagreement between it and thevt100path is an emulation gap made visible.terminal::proc— child processes with a deadline.terminal::driver—SessionDriver, a scenario-facing wrapper overBox<dyn Session>: implementSessionto write a backend, useSessionDriverto write a scenario. Its tests are the integration suitetests/session_driver.rsrather than a unit module.evidence::screenshotandevidence::cast_video— stills and video frames rendered through one renderer rather than two unrelated ones.evidence::dedup— skips re-rendering a screen identical to the step before it.evidence::uploader— a publishing seam with a fallback chain that records which store it fell back from.
Measured agreement, not parity
The step and assertion layers are measured against corpora recorded from the
Python implementation. On those corpora the two runtimes reach 82/115 full
agreement on steps and 124/147 on assertions. That is a layer-level number, not
a product-level one — screen fidelity and whole-recipe execution are outside
it. conformance/README.md in the repository is the authority on the counts and
the divergences; docs/status-and-parity.md
carries the full inventory of what a run still cannot do.
The canonical recipe schema and the example corpus are the contract both
implementations answer to, and they are owned by the Python implementation.
The schema is vendored into this crate at resources/recipe-schema-v1.json
and embedded with include_str!, so load_canonical_schema answers the same
from a published tarball as it does from the repository, and reads no path
outside the crate. python/scripts/check_schema_copies.py holds that copy
byte-identical to the Python package's in CI. The corpus is the shared
conformance/ tree at the repository root and is not vendored.
What this crate does pin is its own generated schema, to a checked-in
snapshot (tests/snapshots/recipe_schema_v1.json, guarded by
tests/schema_snapshot.rs). The guard compares parsed JSON trees, so object
key order is ignored but any structural drift in generate_recipe_schema()'s
output — keywords, numbers, array order, $ref targets — fails the test, and
re-blessing is a deliberate env-var flow (TERM_PROOF_BLESS_SCHEMA=1). It
catches accidental changes to this crate's schema; it does not establish
agreement with the canonical schema. Comparing the two is parity-gate work,
still open — reaching the file was the precondition, not the gate.
Known gaps
terminal::DockerSessionBackendis a stub.terminal::InMemorySessionencodes test-passing rather than PTY semantics:wait_for_textanswers from fixed content and ignores its deadline, andwait_for_idlealways returns true.- Two
presskey mappings (ctrl-[,ctrl-1) are refused here that the Python oracle accepts. - The CLI parses
--video,--diffand--update-baselinesbut does not yet call into the correspondingevidencefunctions; they are reachable as a library API, not from atermproof run.
A recipe also cannot branch on what it observes, and that one is a decision
rather than a gap. A scenario that polls until something renders and acts only
if it did, dismisses an overlay that may or may not appear, or retries a racy
step, belongs in the consumer's own runner — driving a session through
terminal::SessionDriver, using evidence, and building a result::RunResult
itself so parity, before_after and the reporters still apply.
docs/conditional-recipes.md in the repository has the reasoning, a worked
example against the driver, and the reversal conditions.
Package contents
The two differential tests (tests/differential_steps.rs,
tests/differential_assertions.rs) are excluded from the published package —
they replay a corpus that lives at the repository root and is not shipped.
Everything else in tests/ ships, including the schema snapshot
(tests/snapshots/recipe_schema_v1.json) and its guard
(tests/schema_snapshot.rs), so a consumer that runs the published crate's
tests gets the same drift check this repository runs. cargo package --list
is the proof. The differential tests are run from a repository checkout.
Licence
MIT — see LICENSE.