# Testing
## Quality gate
**Required for any change touching a primitive or rasteriser:**
```bash
cargo build --features tui --features gtk
cargo test --features tui
cargo test --features gtk # If GTK runtime is available
cargo clippy --features tui -- -D warnings
cargo clippy --features gtk -- -D warnings
cargo fmt --check
```
If GTK runtime libraries aren't available locally, the GTK feature won't
build — that's a CI concern. Run TUI checks at minimum.
**Mini-app validation:** if the change affects a primitive consumed by
the kubeui demos, verify they still build:
```bash
cd kubeui && cargo build
cd kubeui-gtk && cargo build # if GTK is available
```
## CI matrix (`.github/workflows/ci.yml`, `.github/workflows/macos.yml`, `.github/workflows/fuzz.yml`)
The commands above are the local minimum; CI runs a wider matrix on every
push/PR to `main` or `develop`. This section is kept in sync with the
workflow files as the matrix changes — if you touch either file, update
this list in the same PR.
| Job | Runner(s) | What it covers |
|---|---|---|
| `rustfmt` | ubuntu-latest | `cargo fmt --all --check` |
| `repo-lint` | ubuntu-latest | `cargo run -p quadraui-repo-lint` (`tools/lint`, #1110's meta-test-relocation half — see "What unit tests don't cover" below for the still-open host-font half of #1110) — maintainer-workflow guards (CLAUDE.md/ci.yml wording, `.githooks/` worktree behaviour, example manifest hygiene) that used to be `quadraui/tests/*.rs` files an outside contributor's ordinary `cargo test` would trip over; moved to a plain binary CI runs explicitly instead |
| `Example-test coverage` | ubuntu-latest, PR events only | `tools/example_coverage.py --fail-on-gap` — a new `tui_*.rs` example added by the PR must have a matching `TuiDriver` test |
| `tui (build, test, clippy)` | ubuntu-latest, **windows-latest** (`continue-on-error`, see the job's comments for the rollout plan) | `cargo build`/`test`/`clippy --features tui`; a `win`-feature compile check + arithmetic-only unit tests (`cargo check`/`test -p quadraui --features win`, ubuntu leg only, #538); on the windows-latest leg (all three `continue-on-error`, see the job's comments for the rollout plan): a real build + clippy of the `win_demo` example (`--features win`, #19/#21) and `cargo test -p quadraui --features win` (#24 — the first step that actually *runs* `cfg(target_os = "windows")` code against real Direct2D, via `src/win/testing.rs`'s `HeadlessSurface`); the tier-3 pty smoke suite (`--features tui,terminal --test tui_pty_smoke`); a build of the `tui_terminal` example (`--features tui,terminal`, #483 — `required-features` means the plain `tui`-only build above silently skips it); the `terminal_engine` unit tests (`--features tui,terminal --lib terminal_engine`, #483 — gated on `terminal` alone in `lib.rs`, so nothing else in this matrix compiles them); the conformance matrix, uploaded as an artifact per OS leg; on the windows-latest leg only, a second conformance run under `--features win --test conformance -- --nocapture` (quadraui#722 review) uploads the `win` column's C0 and Tier-1 tables as their own artifacts (`conformance-matrix-c0-win`, `conformance-matrix-win`) — without it, the `win` burn-down grid never left the log of a step whose stdout `cargo test` discards on a pass |
| `gtk (build, test, clippy)` | ubuntu-latest | `cargo build`/`test`/`clippy --features gtk,tui` (both backends, for `cross_backend_parity.rs`); a build of the `gtk_terminal` example (`--features gtk,terminal`, #483, same `required-features` gap as `tui_terminal` above); the gtk+tui conformance matrix, uploaded as an artifact |
| `downstream consumers (compile truth)` | ubuntu-latest | `cargo check --all-targets` against coord-tui and vimcode, both redirected onto this PR's quadraui checkout — see the job's comments for how (#528) |
| `macos (build, test, clippy)` (separate workflow, `macos.yml`) | macos-latest, **`pull_request` gated on a `quadraui/src/**` path filter (#598, widened by #792), plus `workflow_dispatch`/weekly `schedule`**, with a per-ref `concurrency` group cancelling superseded runs | `cargo build`/`test`/`clippy --features macos` — the only place `src/macos`'s tests (including `MacDriver`'s, quadraui#493, and the Tier-1 `backends()` row it adds to `tests/conformance.rs`) run; the Tier-1 conformance matrix (`--features macos --test conformance -- --nocapture`), uploaded as `conformance-matrix-macos` (no Tier-0/C0 artifact yet — `MacBackend` has no C0 driver, see `c0_paint_smoke`'s doc); a Homebrew `gtk4` install followed by `cargo test --features macos,tui,gtk --test cross_backend_parity` (#792 item 3) — the only leg where the macOS twin test in that file (`pipeline_parity_macos_agrees_with_tui_and_gtk_on_logical_state`) actually runs, since the whole file is `cfg(all(feature = "tui", feature = "gtk"))` at the top. The path filter is load-bearing, not incidental (see the workflow's own header comment): a PR that doesn't touch `quadraui/src/**` gets no check at all, but one that does — macOS-specific or not, per #792's fix for the #537 failure mode — gets a *real*, merge-blocking macos-latest build+test+clippy — `#484`'s progress bar. Still weekly (Mondays 06:00 UTC) / manual-dispatch otherwise, for the same "don't let a regression sit undiscovered" reason as before #598 |
| `fuzz` (separate workflow, `fuzz.yml`, quadraui#1130) | ubuntu-latest, **`schedule` (daily 03:00 UTC) + `workflow_dispatch` only — never `push`/`pull_request`** | `cargo +nightly fuzz run <target>` (via `cargo-fuzz`, installed fresh each run) for each target in `quadraui/fuzz/fuzz_targets/`, 5 minutes each, one job per target (`fail-fast: false`) so one crash doesn't hide the others' results. Crash artifacts upload via `actions/upload-artifact` on failure. See "Property tests and fuzzing" below for what's covered and why this never blocks a PR |
Not yet covered by CI at all: real `WinBackend` *rasterisers* running on
Windows — every `Backend::draw_*`/`*_layout` method is still a `todo!()`
stub (see `src/win/backend.rs`'s module docs), so there is nothing yet
for a `windows-latest` test to paint through the trait. #24's
`HeadlessSurface` closes the *infrastructure* gap (a real, GPU-free
Direct2D render target a test can paint into and read pixels back from)
ahead of the rasterisers that will need it — see the `tui` job row above
for the three `continue-on-error` windows-latest steps and the rollout
plan that flips each to blocking once proven stable.
## Property tests and fuzzing (quadraui#1130)
The paint/click round-trips and example-driver tests below prove
*behavioural* correctness against hand-picked scenarios. They say nothing
about the crate's small set of parsers that take **untrusted or foreign
input** — a VS Code theme file downloaded from a marketplace
(`Theme::from_vscode_json` / `text_util::strip_json_comments`), a
markdown document from a chat transcript or README
(`compose::markdown::render_markdown_to_styled`), or whatever bytes a
child process writes into an embedded terminal's PTY (the `vt100` parser
`terminal_engine::TerminalSession` wraps). A hand-picked fixture can only
cover the malformed shapes someone thought to write; these need the
arbitrary-input space instead.
**Two layers, same split as the rest of this doc's CI-vs-local posture:**
1. **`proptest` — in-tree, every PR, bounded.** Each of the files above
has a `#[cfg(test)] mod proptests` block (`src/text_util.rs`,
`src/theme.rs`, `src/compose/markdown.rs`, `src/terminal_engine.rs`,
plus `src/tui/caps.rs`'s `parse_decrqm_reply` and
`src/tui/backend.rs`'s SGR-mouse-report decode) asserting "never
panics" and, where the function has one, a real round-trip/invariant
property — not just the absence of a crash. These run as part of the
ordinary `cargo test` in the quality gate above: hundreds of
generated cases per property, sub-second, no extra tooling.
2. **`cargo-fuzz` — scheduled, unbounded, coverage-guided.** `quadraui/fuzz/`
is a standalone `cargo-fuzz` crate (its own `Cargo.toml`/`Cargo.lock`,
opted out of the root workspace — see that `Cargo.toml`'s comments)
with one libFuzzer target per parser that has a reachable `pub` entry
point: `strip_json_comments`, `theme_from_vscode_json` (round-tripped
through a real temp file, since `Theme::from_vscode_json` only takes a
`Path`), `markdown`, and `vt100_parser` (drives the `vt100` crate
directly — see that target's doc for why). Run via
`.github/workflows/fuzz.yml`, `schedule`/`workflow_dispatch` only,
**never** `push`/`pull_request` — a fuzz run's value is continuous
background exploration on a timescale no per-PR budget affords, and
bolting it onto `ci.yml` would only slow down every PR for coverage
the scheduled run already gives more thoroughly. Run locally with:
```bash
cd quadraui/fuzz
cargo +nightly fuzz run strip_json_comments # or any other target
```
**Why no fuzz target for DECRQM/kitty-keyboard decoding.** `parse_decrqm_reply`
(`src/tui/caps.rs`) is `pub(crate)`, not `pub` — a separate fuzz crate
outside `quadraui`'s own crate boundary cannot reach it without a new
public export, and CLAUDE.md's downstream-impact discipline (see the root
`CLAUDE.md`'s "Downstream consumers" section) argues against adding `pub`
surface whose only consumer is a fuzz harness. Its `proptest` coverage in
`src/tui/caps.rs::proptests` is the whole story for that parser — an
in-crate property test has no such visibility restriction.
**A property test already found a real bug this way.** The initial,
unbounded `vt100_parser_never_panics*` properties (`rows`/`cols` with no
floor) found two upstream `vt100` 0.16.2 panics — `rows < 2` and,
separately, `cols == 1` with certain multi-byte input — that a real host
resizing an embedded-terminal pane down to a single row or column could
have hit in production. `terminal_engine.rs`'s `MIN_VT100_ROWS`/
`MIN_VT100_COLS` consts now floor every caller-supplied dimension before
it reaches `vt100::Parser::new`/`Screen::set_size`, pinned by
`spawn_with_a_single_row_does_not_panic` and its three siblings in that
file's `mod tests`. This is the point of property testing a parser you
don't own: it doesn't have to be *your* bug for it to be worth defending
against.
## Coverage taxonomy
Three bug classes, three test shapes. An agent picking up an issue
should map the work to the relevant rows and add tests accordingly —
no per-issue restatement needed.
| Bug class | Test shape | Lives in |
|---|---|---|
| **Coordinate drift** between paint and click | Paint/click round-trip — paint into the backend's headless surface, find a painted glyph, hit_test that exact coordinate, assert the hit identifies the painted element. | `tui/<name>.rs::tests` and `gtk/<name>.rs::tests`. |
| **Consumer-side click-routing drift** | Consumer-state round-trip — paint, simulate the consumer's click handler, assert the host's state mutation matches the painted UI. | Adjacent to the consumer pattern. Template: `tui::multi_section_view::tests`. |
| **State-derived paint geometry** | Painted-indicator test — set state to a known value, paint, find the indicator in the buffer/surface, assert it lands at the position the formula predicts. | Same module as the rasteriser. |
| **Example / app-wiring drift** | Example-driver round-trip — drive the *whole* `AppLogic` through the headless driver, script real `UiEvent`s, assert on the re-rendered screen. Catches mis-routed handlers, missing `Reaction::Redraw`, stale state — none of which (1)–(3) can see. | `tests/tui_example_driver.rs` (TUI, `TuiDriver`), `tests/gtk_example_driver.rs` (GTK, `GtkDriver`), and `tests/macos_example_driver.rs` (macOS, `MacDriver` — quadraui#493, still thin: `PipelineApp`/`DataTableApp` only). |
Every primitive needs (1). Primitives with consumer-pattern recipes
need (2). Primitives with state-derived indicators need (3). Every
runnable example should have at least one (4) covering its core
interaction.
**Each test must be empirically verified by mutation.** Break the
contract (zero out the offset, swap a +/-, paint at the wrong y),
observe at least one test fail, restore. A green test that doesn't
catch its bug class is theatre.
## Acceptance bar for new code
**Every PR that adds or changes a primitive (`<name>.rs`) or an example (`examples/tui_*.rs` / `examples/gtk_*.rs` / `examples/macos_*.rs`) must include the matching test from the coverage taxonomy above** — the paint/click round-trip for a primitive, the example-driver round-trip for an example (TUI via `tests/tui_example_driver.rs` / `TuiDriver`, GTK via `tests/gtk_example_driver.rs` / `GtkDriver` — both landed, #301 GD-1..GD-3; macOS via `tests/macos_example_driver.rs` / `MacDriver` — quadraui#493, still thin coverage of the 16 `macos_*` examples). **A PR missing its test is rejected at review.** This is enforced by the adversarial reviewer, which reads the project rules in [`CLAUDE.md`](../../CLAUDE.md) (see *"Demos are mandatory for visual features"*).
Tests must use the **high-level driver API** — `find("text")` to locate a painted target, then `click(x, y)` with the coords it returns, plus `screen_contains()`, `press()`, `type_char()`. **Hardcoded coordinates are brittle and out of policy** — locate, don't guess. A coordinate that's correct today silently rots the first time padding, a label, or a layout metric changes.
**Checking the rule is measurable, not just stated:** `tools/example_coverage.py` (repo root) prints a matrix of every `examples/tui_*.rs` / `examples/gtk_*.rs` against whether its `AppLogic` struct is wired into the matching example-driver test file, and exits non-zero if any example is missing coverage (#310).
```bash
tools/example_coverage.py
```
**This is also a CI gate, not just a local check (#311):** the `Example-test coverage` job in `.github/workflows/ci.yml` runs the same tool with `--fail-on-gap` on every PR, which fails the check only when a `tui_*.rs` example *added by that PR* lacks coverage — pre-existing gaps (most of GTK, and the TUI examples still mid the #B1-#B5 backfill) print in the matrix but don't fail the run, and newly-added `gtk_*.rs` examples are excluded entirely pending `GtkDriver` (#301). CI diffs against the PR's actual base commit (`github.event.pull_request.base.sha`); to approximate that locally, diff against `origin/develop` — this repo's branching policy merges feature PRs into `develop` (`main` only moves via release merges from `develop`), and `--base` defaults to `origin/develop` for exactly this reason:
```bash
tools/example_coverage.py --fail-on-gap --base origin/develop
```
## Example-driver tests (end-to-end, in-process)
`quadraui::tui::testing::TuiDriver` drives a whole `AppLogic` impl — the
same type the `tui_*` examples instantiate — through the real
event → `handle` → `render` path, against ratatui's in-memory
`TestBackend`. No TTY, no pty: deterministic and `cargo test`-native.
```rust
let mut d = TuiDriver::new(PipelineApp::new(), 100, 30);
let (x, y) = d.find("Go").unwrap(); // locate the painted action button
d.click(x, y); // MouseDown in cell coords — no escape math
assert!(d.screen_contains("stage 3")); // click round-tripped paint→hit_test→handle→render
```
- **Why it's distinct from the round-trips above.** Those test one
`draw_*` fn on a hand-built struct; this tests the example's wiring.
- **No drift from production.** `render`/`dispatch` call the same
`tui::run::render_frame` / `dispatch_event` the live runner uses, so
the test path renders + pre-processes events (text selection, Ctrl-C)
identically to `tui::run`.
- **Mouse is a `UiEvent::MouseDown` in backend coordinates** — no SGR
escape-sequence math, the ergonomic win over a pty runner for
click-heavy primitives.
- **Drags work too.** `mouse_down`/`mouse_move`/`mouse_up`/`drag` route
through `TuiBackend::translate_injected` — the same `apply_dispatch` +
`DragState` layer `wait_events` uses — so a scripted drag exercises the
real selection/drag machinery. E.g. drag over a registered text region
→ `TextSelectionChanged`, then `ctrl_char('c')` → `TextCopied` (see
`panel_drag_selects_text_and_ctrl_c_copies_it`). (Scrollbar thumb-drag
isn't wired into any TUI example yet — `apply_dispatch` passes no
scroll surfaces — so there's nothing to drive there until a consumer
adopts it; the offset math is unit-tested in `dispatch.rs`.)
- **Generalizes across backends.** Because `AppLogic` is
backend-neutral, `GtkDriver` (below) feeds identical scripted events to
the identical app and snapshots the Cairo surface — true cross-backend
parity from one event script.
**Limitation:** the driver renders into a `TestBackend` buffer, so it
does *not* exercise real ANSI/escape emission — terminal-protocol bugs
(raw-mode setup, escape parsing, SGR mouse decoding; e.g. #293) are out
of scope and need a pty-based smoke test instead — see Tier-3 below.
## Tier-3 pty smoke (real terminal-protocol black box, quadraui#302)
`tests/tui_pty_smoke.rs` closes the exact gap the Limitation above
calls out. It spawns the *actual* example binary
(`cargo run --example <name> --features tui`) inside a real
pseudo-terminal via [`portable-pty`](https://docs.rs/portable-pty)
(the same crate `terminal_engine.rs` uses for the embedded-terminal
primitive) and parses the emitted byte stream with
[`vt100`](https://docs.rs/vt100) into a screen model — no `TestBackend`,
no injected `UiEvent`s. Keystrokes and raw SGR mouse-report bytes are
written to the pty's stdin exactly as a real terminal would deliver
them; assertions read back the emulated screen (`vt100::Screen::contents`).
```bash
cargo test -p quadraui --features tui,terminal --test tui_pty_smoke
```
- **What it catches that `TuiDriver` can't.** Raw-mode / alternate-screen
setup actually working on a real pty, the ANSI escape stream `tui::run`
emits parsing back to the expected screen, and SGR mouse-report
round-trips (the #293 class — motion events leaking their raw escape
bytes into a focused text input instead of decoding to a mouse event).
- **The PTY master must behave like a terminal, not a dumb pipe.**
`ratatui`'s crossterm backend queries the cursor position
(`ESC [ 6 n`) once during `Terminal::new()` and treats a missing reply
as fatal — a real terminal always answers this. `PtyExample`'s
background reader thread plays that minimal terminal-emulator role,
replying `ESC [ row ; col R` from the `vt100` parser's tracked cursor
position. Skipping this makes every example fail before it ever
renders (confirmed empirically — see the harness's module doc).
- **Deliberately thin — a handful of representative examples**
(`tui_pipeline`, `tui_chat`, and `tui_split`/`tui_no_mouse` for the
`no_mouse` module below), not broad coverage. This is wiring/
integration confidence for the terminal-protocol layer; `TuiDriver`
(Tier "example / app-wiring drift" above) remains the primary,
deterministic tool for everything else — coordinate drift, click
routing, state-derived paint, and per-example behavior all stay
covered there.
- **Runs as a real CI gate**, not an operator-only tier like the GTK
live-app smoke below. A pty is a kernel device, not a display server —
no Xvfb, no compositor, works headlessly anywhere `openpty` does.
- **Byte-exact escape-sequence assertions are `#[cfg(unix)]`** — the
`sgr_color_depth` module (quadraui#826's SGR colour-depth fixtures),
which reads the raw master-side bytes via `raw_contains`/`wait_for_raw`
rather than the parsed screen. On Unix, `openpty(3)` is a kernel byte
pipe: what the child writes is what the master reads, so those bytes
really are "what a terminal would receive from our app". On Windows,
`portable-pty` is backed by ConPTY (`CreatePseudoConsole`), which puts
a whole conhost terminal emulator in between — the master reads
conhost's re-serialised repaint of its own text buffer, with its own
SGR encoding and batching, not the child's bytes. Content-level
assertions (`wait_for`/`screen_text`) survive that round trip and stay
cross-platform; byte-exact ones cannot. **If you add a raw-byte
assertion, put it in that module.** The platform-independent half of
the same claim — that a painted cell is actually quantised to the
detected depth — lives in `src/tui/color.rs`'s
`pipeline_theme_pair_quantises_through_a_real_draw_at_every_depth`,
which does run on Windows.
- **Fixtures that control input *byte boundaries* are `#[cfg(unix)]` too**
— same ConPTY reason, mirrored onto the input direction, and the gate
belongs on the test rather than on a module here.
`tui_chat_escape_glued_to_sgr_motion_in_one_write_does_not_leak`
(quadraui#293) works only because it can glue a lone `ESC` and an SGR
mouse report into a single `write()`, which is the input crossterm's
Unix event source then has to disambiguate
(`parse_event(buffer, input_available)` — #293's actual mechanism). On
Windows crossterm never parses bytes at all: `event::source::windows`
reads `INPUT_RECORD`s through the console API, and ConPTY has already
re-parsed the master-side bytes with conhost's own VT input state
machine before the child sees anything. The byte boundaries the test
constructs therefore do not survive to the code under test, and a
failure there reports on conhost, not on quadraui. **Rule of thumb: if
a pty fixture's premise is "these exact bytes, in this exact grouping"
— in either direction — gate it `#[cfg(unix)]` and cover the logic
itself with an in-crate unit test that runs everywhere** (here:
`src/tui/backend.rs`'s `recover_leaked_sgr_mouse_fragments` tests).
Content-level pty assertions ("typing works", "the report didn't leak
into the transcript") stay cross-platform and still run on
windows-latest.
## GtkDriver example-driver tests (end-to-end, in-process)
`quadraui::gtk::testing::GtkDriver` is the GTK twin of `TuiDriver` (#301,
GD-1..GD-3, #446-448): it drives a whole `AppLogic` impl — the same type
the `gtk_*` examples instantiate — through the real
event → `handle` → `render` path, against a headless
`cairo::ImageSurface`. No `gtk::init`, no `Application`, no
`GdkDisplay`, no Xvfb: deterministic and `cargo test --features
gtk`-native, runs anywhere the `gtk4`/`cairo`/`pangocairo` crates link
(incl. dellserver, no display server needed).
```rust
let mut d = GtkDriver::new(PipelineApp::new(), 800, 480);
let (x, y) = d.find("Go").unwrap(); // locate the painted action button
d.click(x, y); // MouseDown in pixel coords
assert!(d.screen_contains("stage 3")); // click round-tripped paint→hit_test→handle→render
```
- **No drift from production.** `render`/`dispatch` call the same
`gtk::run::render_frame` / `dispatch_event` the live `quadraui::gtk::run`
runner uses, and `click`/`drag` route through the same
`dispatch_click` / `dispatch_mouse_drag` / `dispatch_mouse_up` the live
handlers use, so the test path behaves identically to production.
- **Assertion API mirrors `TuiDriver`'s shape** — `find("text")` /
`screen_contains()` / `press_named()` / `type_char()` — backed by the
`(text, bounds)` map recorded at paint time (GD-2, #447) plus pixel
readback via `pixel(x, y)` for colour/geometry assertions Pango text
search can't express.
- **Core smoke set + cross-backend parity** live in
`tests/gtk_example_driver.rs` (GD-3, #448) and
`tests/cross_backend_parity.rs` — the latter runs one scripted event
sequence through both `TuiDriver` and `GtkDriver` and asserts they
reach the same logical state.
### Out of scope / limitations
The offscreen driver rasterises the `draw_*` primitives through
`GtkBackend::enter_frame_scope`; it does **not** instantiate the real
`Application`/`ApplicationWindow` or talk to a compositor. Therefore:
- **Toplevel-window-sizing bugs** (e.g. #437 "opens with tiny/broken
window") are out of its reach — window geometry is negotiated with the
real WM/compositor, which this driver never touches.
- **Real clipboard/paste** is out of its reach — quadraui's GTK backend
uses `arboard`, which hits the X11/Wayland selection; the headless
surface has no selection owner to talk to.
- **Raw GDK signal delivery** is out of its reach — raw keycode
translation (`gdk_key_to_uievent`), IME composition, and actual
`EventController` wiring are bypassed; `GtkDriver` injects `UiEvent`s
directly, the same unified boundary `TuiDriver` uses.
- **Terminal-protocol / real-GL rendering paths** likewise stay with the
live app.
These all belong to **GD-5 (#450)** — live-app headless smoke that runs
the *real* window under Xvfb/Broadway — or a real-display oracle
(precision/elitebook) for cases GD-5 still can't reach.
### Cross-backend example tests: shared bodies, per-backend adapters
Now that `GtkDriver` has landed (#301, GD-1..GD-3), example-driver tests
should **not** be duplicated per backend. The split is hybrid — ~80%
shares, ~20% is irreducibly backend-specific:
| Layer | Shared? | Why |
|---|---|---|
| The `AppLogic` under test | yes — identical | backend-neutral by design |
| The event script (`press`/`click`/`drag`) | yes — identical | `UiEvent` is the unified boundary |
| Logical / state assertions | yes — identical | same app → same state on every backend |
| Reading "the screen" | **no** | TUI = character-cell grid (string search); GTK = Cairo pixel surface (Pango text map) |
| Coordinate units | **no** | TUI = cells (`line_height 1`); GTK = pixels (`line_height ~16`) |
Abstract the two backend-specific rows behind a small trait, write each
test body **once** as a generic fn, and run it against both drivers:
```rust
trait ExampleDriver {
fn press_named(&mut self, k: NamedKey);
fn type_char(&mut self, c: char);
fn click_text(&mut self, needle: &str); // locate text → click center (native coords)
fn drag_text(&mut self, from: &str, to: &str);
fn screen_has(&self, needle: &str) -> bool;
fn exited(&self) -> bool;
}
fn check_pipeline_click<D: ExampleDriver>(mut d: D) { // written ONCE
d.click_text("Go");
assert!(d.screen_has("stage 3"));
}
#[test] fn tui() { check_pipeline_click(TuiDriver::new(PipelineApp::new(), 100, 30)); }
#[test] fn gtk() { check_pipeline_click(GtkDriver::new(PipelineApp::new(), 800, 480)); }
```
**Rules for shared bodies:**
1. **Locate by semantics, never literal coordinates.** Use
`click_text("Go")` / `drag_text(a, b)`, not `click(12.0, 3.0)` — each
driver resolves text to its own native position (TUI scans the cell
grid; GTK queries the `(text, bounds)` map it records at paint time),
so bodies stay unit-agnostic. A test that hard-codes cell/pixel
numbers will not port.
2. **Assert on logic/text, not pixels, in shared bodies.** `screen_has`
works on both (cell-grid search vs Pango text map).
**Parity test (free bonus):** run the same script on both drivers and
assert they reach the same logical state — the strongest cross-backend
guarantee:
```rust
fn pipeline_parity<D: ExampleDriver>(mut d: D) -> Vec<bool> {
d.click_text("Go");
vec![d.screen_has("stage 3"), d.exited()]
}
assert_eq!(pipeline_parity(tui), pipeline_parity(gtk));
```
**Irreducible per-backend residue (keep separate, by design):** exact
pixel colours, 1px borders, font rendering, double-width glyph handling.
These genuinely differ — TUI keeps cell-style assertions; GTK gets
pixel/Pango checks. Don't try to share these.
**Realized in `tests/cross_backend_parity.rs`:** the `ExampleDriver`
trait sketched above is implemented for both `TuiDriver` and
`GtkDriver`, and `pipeline_parity_tui_and_gtk_agree_on_logical_state`
runs one shared script body against both. Per-example TUI/GTK suites
(`tests/tui_example_driver.rs`, `tests/gtk_example_driver.rs`) stay
separate for now — only the cross-backend parity test currently uses
the shared-body pattern; migrating the full per-example suites onto it
is follow-up work, not required by #301.
### `FrameInventory`: the portable paint-inventory contract (quadraui#490)
Before #490, "what did you paint and where" was backend-specific and
leaked platform types into test bodies: TUI drivers scanned the cell
grid with `find`/`find_bounds`, GTK read a `(text, bounds)` map, macOS
had nothing, and a test that wanted to relate two painted things
("is X left of Y?", "is X inside the sidebar zone?") had no
backend-neutral way to ask.
[`quadraui::testing::FrameInventory`] (`quadraui/src/testing.rs`, no
`tui`/`gtk`/`macos`/`win` feature gate) is that contract — the same
shape `#322` proposes for quadraweb's id→rect map:
```rust
pub struct TextRun { pub text: String, pub bounds: Rect }
pub struct ZoneRec { pub id: WidgetId, pub bounds: Rect }
pub struct FrameInventory { pub text_runs: Vec<TextRun>, pub zones: Vec<ZoneRec> }
```
`ConformanceDriver::inventory()` returns one per frame. `text_runs` is
recovered from each backend's existing paint output with no per-widget
opt-in (TUI's wide-char-aware cell-grid scan; GTK's `painted_text` map,
fed by every Pango draw via `gtk/painted_text.rs`'s choke point).
`zones` is populated from [`Backend::register_zone`] calls made during
render — currently `AppShell::render` registers one zone per
activity-bar item (keyed by that panel's real `WidgetId`) plus one
`app-shell:`-prefixed zone per shell-chrome region (`activity-bar`,
`sidebar-header`, `sidebar-content`, `main-content`, `status-bar`,
`command-line`, `divider`, `bottom-panel`, `title-bar`, `window`); other
primitives haven't been wired to call `register_zone` yet and simply
contribute no zone, never a wrong one.
`FrameInventory` carries a small relational assertion vocabulary — the
structural fix for CLAUDE.md's no-hardcoded-coordinates rule, since
every one of these is computed on the `Rect`s the inventory already
reports, in the backend's own units, so a shared test body never writes
a coordinate itself:
```rust
let inv = driver.inventory();
inv.screen_has("EXPLORER"); // any text run contains this substring
inv.absent("SOURCE CONTROL"); // no text run does
inv.count("Settings"); // how many runs do
inv.left_of("E", "EXPLORER"); // a's right edge <= b's left edge
inv.above("EXPLORER", "content"); // a's bottom edge <= b's top edge
inv.same_row("E", "S"); // vertical ranges overlap
inv.inside("content", &WidgetId::new("app-shell:sidebar-content"));
```
`left_of`/`above`/`inside` are proven to agree across `TuiDriver` and
`GtkDriver` for the `AppShellDemo` shell_app fixture in
`tests/cross_backend_parity.rs` — see
`frame_inventory_relations_agree_tui_and_gtk`.
**Backend checklist:** every backend driver that implements
`ConformanceDriver` MUST implement `inventory()` and populate
`text_runs` from its real paint output (not a stub). Populating `zones`
is incremental (call `Backend::register_zone` from whichever composers/
primitives you want zone-testable) but the field must exist on the wire
from day one — declare it empty, never omit it, so callers never need a
breaking change when a new zone source lands.
## Conformance tiers (C0–C4) and the scenario suite (quadraui#491)
The conformance suite is Tier-4 generalized: **declarative** scenarios,
run against **every** registered backend, producing a scenario × backend
matrix. It complements — does not replace — the tiers above:
`docs/SMELL_AUDIT_2026-07.md` §6.7 spells out the relationship (Tiers 1–3
unchanged; the pty smoke #302 unchanged; `GtkDriver`/GD-5 division of
labour unchanged).
**A new conformance fixture does not waive the per-example Tier-1
obligation.** When a Tier-1 scenario's fixture is a brand-new `tui_*`
example (CLAUDE.md's "Demos are mandatory for visual features" /
"Every TUI example also ships an automated black-box test"), that example
still needs its own `TuiDriver` cluster in `tests/tui_example_driver.rs` —
same as any other new `tui_*` example, conformance scenario or not. The
two mechanisms check different things: the Tier-1 driver test is a
hand-written, backend-specific regression net over that one example's
behaviour; the Tier-4 scenario is the same interaction expressed once and
proven identical across every registered backend. Neither substitutes for
the other. (`tui_modal_occlusion` / `ModalOcclusionDemo` — the fixture
behind `dialog.blocks_click_through` — ships both: see the
`modal_occlusion_*` tests in `tests/tui_example_driver.rs`.)
### The tiers (a new backend's burn-down checklist)
- **C0 — Boot (day one).** Construct the backend headless;
`begin_frame`/`end_frame`; sane viewport; draw *every* primitive once
with a canned descriptor — no panic **and** non-empty `text_runs()` for
text-bearing primitives. This turns the silent no-op `Backend` defaults
into visible red/green immediately. *(Auto-generated per primitive —
quadraui#492.)* `win` has a column here too (quadraui#722), registered
burn-down rather than blocking — see *Burn-down columns* below, which now
covers both `c0_paint_smoke`'s per-primitive grid and
`conformance_matrix`'s per-scenario rows.
- **C1 — Interaction core (mandatory for "complete").** The Tier-1
scenarios: click routing, keyboard focus, modal occlusion,
scroll-under-cursor, split relayout, text-selection drag + copy, tab
close, menu open/navigate/activate, palette type/pick, toast dismiss,
editor click-to-caret. **This is what ships today** — see the ten
scenario files under `tests/conformance/scenarios/`.
- **C2 — Event parity.** For each required `UiEvent` variant, a
per-backend native-injection recipe proving it is emitted. Required
(windowed backends — GTK/Win/macOS):
Key/Char/MouseDown/Up/Moved/Scroll/DoubleClick/WindowResized/
WindowClose/Accelerator/ClipboardPaste/TextCopied. TUI has no OS
window, so `WindowClose` doesn't apply there — see
`quadraui/docs/decisions/DECISIONS.md` D-010. Optional (declare, don't fake):
FilesDropped, MouseEntered/Left, DpiChanged, native menu events. The
full required/optional-per-backend table, with current pass/fail per
cell, lives in `quadraui/docs/BACKEND.md`'s "UiEvent emission matrix".
`tests/conformance/c2.rs` (quadraui#501) is the mouse/key/scroll/
resize slice of this tier's harness on TUI+GTK; the rest of the
required list is tracked follow-up (see D-010's *Follow-ups*).
- **C3 — Platform services.** Clipboard round-trip (headless where
possible), capability-honest dialogs and notifications.
- **C4 — Native residue (never shared).** Exact colours, font rendering,
wide-glyph pixels, live-window smoke. GD-5 stays exactly as it is.
### Running it
```sh
cargo test -p quadraui --features tui --test conformance -- --nocapture
cargo test -p quadraui --features gtk,tui --test conformance -- --nocapture
```
The Tier-1 matrix prints on every run and is also written to
`$CARGO_TARGET_DIR/conformance-matrix.txt`; the Tier-0 grid
(`c0_paint_smoke`) does the same to `$CARGO_TARGET_DIR/conformance-matrix-c0.txt`
(quadraui#722) — two files, not one, because both tests can run
concurrently in the same `cargo test` invocation and a shared path would
let whichever finished last silently clobber the other's artifact. CI
uploads both as artifacts from the `tui` and `gtk` jobs (plus a
`win`-only rerun of each on the windows-latest leg — see the CI table
above). For a backend that doesn't exist yet (Windows, macOS) **that
artifact is the implementation checklist**.
```text
Conformance matrix (scenario × backend)
scenario tier tui gtk
-------------------------------------------------
pipeline.click_advances_stage 1 pass pass
dialog.blocks_click_through 1 pass pass
...
```
### Burn-down columns: registered, reported, not gating (quadraui#708)
A backend whose rasterisers are still landing needs a column *before* it
can pass one — otherwise "the artifact is the implementation checklist"
is impossible to satisfy, because adding the column would red every PR on
that platform's CI leg. `runner::Gating` separates the two decisions:
- `BackendReg::register::<F>(name)` — **blocking.** A `FAIL` cell fails
`conformance_matrix`. The normal posture (`tui`, `tui-vt100`, `gtk`,
`macos`).
- `BackendReg::register_burn_down::<F>(name)` — **reported, not gating.**
Cells render identically in the matrix and the detail block, the
artifact still carries them, and a legend under the table names every
non-gating column so a reader can never mistake one for a regression.
Today only `win`, which has no painted-text-run recording yet
(`WinDriver::find`/`find_bounds` are honest `None` stubs), so every
text-locating step reports "not painted".
Burn-down is self-expiring, not a permanent exemption: if a burn-down
column runs at least one scenario and fails *none* of them,
`conformance_matrix` fails with "promote it to `BackendReg::register`".
An all-`skip` column doesn't count as having run — a backend that
declares no capabilities can't earn promotion by never attempting
anything. `every_asserted_zone_is_registered_by_every_backend` skips
burn-down columns for the same reason: a backend with no shell chrome yet
registers no zones, and that gap is already visible as its `FAIL` rows.
**`c0_paint_smoke` (quadraui#722) shares the same `runner::Gating` enum**,
applied to its per-primitive grid instead of `conformance_matrix`'s
per-scenario rows: each column in that test's local `Column` struct carries
a `gating` field, `win`'s is `Gating::BurnDown`, and its `PANIC`/`FAIL`
cells are printed — case names and all, via a "non-gating C0 gap(s)"
line separate from the blocking gap list — but excluded from the
assertion that fails the test. The same legend
(`runner::burn_down_legend`) prints under this tier's table too, and the
same self-expiring promotion check applies: a burn-down column with zero
failing cases fails `c0_paint_smoke` until it's moved from
`Gating::BurnDown` to `Gating::Blocking` in that test's `columns.push`
calls. Unlike the scenario suite there is no capability-skip concept at
this tier — every case in `c0::CASES` always runs — so "ran" is not part
of that check, only "ever failed".
### TUI: two observers, one row each (quadraui#555)
Every tier above — including the conformance suite's own `"tui"`
column — observes the TUI backend at **draw time**: `TuiDriver` reads
`TestBackend`'s retained `ratatui::buffer::Buffer` straight back after
`app.render` returns. That answers "what did the rasteriser ask for?" It
cannot answer "what would a real terminal actually show?", and for a
cell-grid backend those can diverge — a double-width (CJK/emoji) glyph's
reserved continuation cell, a malformed buffer, or an ANSI-encoding bug
can each cause the two to disagree, and #302's own module doc names the
general form: `TestBackend` "never touches a real TTY, so
terminal-protocol bugs are invisible to it."
```sh
cargo test -p quadraui --features tui,terminal --test conformance -- --nocapture
```
adds a **second** TUI column, `"tui-vt100"`
(`quadraui::tui::vt_testing::TuiVtDriver`): the exact same `AppLogic`
fixture, replayed through the exact same production render path
(`tui::run::paint_frame`), but painted through
`ratatui::backend::CrosstermBackend` into a real ANSI byte stream and
read back with [`vt100`](https://docs.rs/vt100) — the same crate
Tier-3's `tui_pty_smoke.rs` uses, minus the OS pty: `TuiVtDriver` drives
its fixture with scripted `UiEvent`s exactly like `TuiDriver`, never real
keystrokes, so there is no terminal *input* to decode and therefore
nothing that needs a live pty to answer a real terminal's queries (see
`quadraui::tui::vt_testing`'s module doc for exactly which query that
would be and how `TuiVtDriver` avoids it — `CrosstermBackend::size()`
reads `/dev/tty`, unrelated to whatever `Write` sink the backend wraps).
The result is deterministic and needs no subprocess, matching what makes
`TestBackend`-based `TuiDriver` fast, just with a real ANSI byte stream —
not a same-process buffer read — in the loop. `tui_pty_smoke.rs`'s real
pty remains the tier that also covers terminal-*input* protocol (raw-mode
setup, SGR mouse decoding); `TuiVtDriver` is scoped to output fidelity.
**Not every scenario pays for it.** A `.scn.json` file only runs against
`"tui-vt100"` if it declares `"text_fidelity": true` (`schema.rs`) — the
matrix has no cell at all (`-`, not `skip`) for a scenario that didn't
opt in, so the bulk of the suite stays `TestBackend`-only. Reach for the
opt-in when a scenario's assertions are actually about text fidelity,
wide (double-width) characters, or cursor placement — most scenarios
aren't and shouldn't pay the extra render.
```text
scenario tier tui tui-vt100
--------------------------------------------------------
tabbar.wide_label_text_fidelity 1 pass pass
tabgroup.close_tab 1 pass -
```
`tests/conformance/scenarios/tabs/tabbar.wide_label_text_fidelity.scn.json`
is that worked example — a `TabBar` whose active tab's label contains a
CJK filename mid-string (`" 1: 日本語.rs "`, quadraui#554's original
report), asserted with the ordinary `assert_screen_has`/`assert_left_of`
vocabulary against both observers unmodified. Building it against
`"tui-vt100"` is what caught quadraui#555's own finding: `TuiDriver`'s
own `screen()`/`screen_has` — the direct backer of `assert_screen_has` —
walked the buffer one column at a time and so inserted a phantom space
after every double-width glyph (the glyph's reserved continuation cell,
which is always blank by construction and never actually reaches
`TestBackend`'s retained buffer, since `Buffer::diff` skips emitting it).
That made the exact same painted content read as *absent* through
`screen_has` while `find_bounds`/`inventory` — already wide-char aware
since quadraui#488 — found it painted as one contiguous run. `TuiDriver`
now strides `screen()` the same way, so both observers agree; see
`tui::testing::screen_is_wide_char_aware_for_adjacent_cjk_glyphs` for the
regression guard. The tab-bar *measurement* bug quadraui#554 itself was
filed against was already fixed at the source (`display_width`-based tab
budgeting in `tui/backend.rs`) before this tier landed, so both observers
already agreed there — this file's value ended up being the `screen()`
gap it caught along the way, which is the general principle #555 argues
for: a second, more-realistic observer earns its keep exactly by
occasionally disagreeing with the first.
### Adding a scenario = one JSON file, no Rust
Drop a `*.scn.json` under
`quadraui/tests/conformance/scenarios/<area>/`. The runner discovers
files from disk, so nothing registers it. The file stem must equal the
scenario's `id` (the suite asserts this, so a matrix row is greppable
back to its file).
```json
{
"id": "pipeline.click_advances_stage",
"fixture": "pipeline_app",
"tier": 1,
"viewport": { "cols": 100, "rows": 30 },
"requires": [],
"steps": [
{ "assert_absent": "stage 3" },
{ "press": "Right" },
{ "click_text": "Go" },
{ "assert_screen_has": "stage 3" },
{ "type_char": "q" },
{ "assert_exited": true }
]
}
```
Steps, one key each (`quadraui/tests/conformance/schema.rs`):
| Act | Assert | Document |
|---|---|---|
| `press`, `type_char`, `type_text`, `ctrl_char` | `assert_screen_has`, `assert_absent`, `assert_count` | `note` |
| `click_text`, `click_text_at` | `assert_left_of`, `assert_above`, `assert_inside` | |
| `drag_text`, `scroll_at` | `assert_exited` | |
**There is no numeric coordinate field anywhere in the schema**, and
serde rejects unknown keys, so `{"click_at": {"x": 12, "y": 3}}` fails to
deserialise. Hardcoded coordinates aren't discouraged — they're
unrepresentable (`schema.rs::numeric_coordinate_steps_are_unrepresentable`
is the executable form of that claim). The only numbers in the schema are
unit-free counts: `tier`, `viewport.cols`/`rows`, `scroll_at.lines`,
`assert_count.count`.
Two gotchas when writing needles:
- `assert_screen_has` / `assert_absent` match a whole painted **row** on
TUI, but the relational assertions (`assert_left_of`, `assert_above`,
`assert_inside`) match a single **text run**, which TUI splits at
spaces. So `assert_screen_has: "OPEN EDITORS"` works while
`assert_left_of: {a: "OPEN EDITORS", …}` does not — use a
single-token needle (`"EDITORS"`) for relational assertions.
- How *many* rows fit is a backend detail (TUI counts cells, GTK divides
pixels by line height). Assertions should name items that are on-screen
on every backend, not rely on a specific row count.
### Adding a backend = one driver impl + one registration line
`tests/conformance.rs`:
```rust
struct MyFactory;
impl runner::DriverFactory for MyFactory {
fn make<A: AppLogic + 'static>(app: A, vp: LogicalViewport) -> Box<dyn runner::DynDriver> {
Box::new(my::testing::MyDriver::new_fixture(app, vp))
}
}
const MY_CAPS: &[&str] = &["mouse", "scroll", "drag", "text_selection"];
// … and, in `backends()`:
regs.push(BackendReg::new("my", MY_CAPS, fixtures::build::<MyFactory>));
```
Everything else — the fixture registry, the step interpreter, the
matrix — is backend-agnostic. `DynDriver` is the object-safe view of
`ConformanceDriver` (which is `Sized`, so it can't be a trait object);
one blanket impl covers every present and future driver.
### Capabilities: a skip must name what's missing
Each backend declares a capability list. A scenario whose `requires`
mentions a capability the backend hasn't declared is **skipped, with the
missing capability printed in the matrix detail block**. A backend
therefore cannot quietly not-run a scenario: either it declares the gap
up front, or the scenario runs and fails. Silence is impossible.
### Zone-backed assertions: what is registered today
`assert_inside` is the one step that does **not** work off painted text —
it needs a `WidgetId`-keyed rect, which a paint site must volunteer by
calling `Backend::register_zone` during the frame. That is opt-in per
paint site, so the set of assertable zone ids is small and explicit.
Registered today, by `AppShell::render` (via `register_chrome_zones`) and
nothing else:
| Zone id | Registered when |
|---|---|
| `app-shell:window`, `app-shell:activity-bar`, `app-shell:main-content` | always |
| `app-shell:title-bar`, `app-shell:sidebar-header`, `app-shell:sidebar-content`, `app-shell:divider`, `app-shell:bottom-panel`, `app-shell:command-line`, `app-shell:status-bar` | when that region exists this frame |
| the panel's own id, e.g. `panel:explorer` | one per activity-bar item |
Both `TuiDriver::inventory` and `GtkDriver::inventory` forward whatever
the frame registered, so a zone available on one backend is available on
both; `tests/cross_backend_parity.rs` asserts that for
`app-shell:sidebar-content` specifically.
**Naming a zone no paint site registers makes the step unsatisfiable, not
flaky.** `FrameInventory::inside` returns `false` for an unregistered
zone exactly as it does for a run that landed outside a registered one,
so the matrix alone can't tell the two apart. Two things close that gap:
`every_asserted_zone_is_registered_by_every_backend` (in
`tests/conformance.rs`) fails with "backend never registers this id" and
a list of what *is* registered, and the `assert_inside` failure text
itself distinguishes never-registered / never-painted / outside-the-rect.
Before writing an `assert_inside` against a new primitive, wire its
`register_zone` call first.
### Known coordinate-free gaps
Two Tier-1 behaviours from the audit's C1 list can't be expressed
coordinate-free yet, and are *not* faked with a literal:
- **Split-divider drag.** The `SplitTree` divider (not the shell's
sidebar divider, which *is* registered as `app-shell:divider`) has no
text on GTK — it's a filled rect — and no `split.rs` paint site calls
`register_zone`, so there is nothing for `drag_text` to name.
`split.direction_toggle_relayout` covers split *relayout* relationally
instead. Unblocked by registering split-divider zones and adding a
zone-anchored drag step.
- **Editor click-to-caret** (`editor_col_at_x`) — needs a caret position
read-back on `ConformanceDriver`.
Two related paint/hit-test rounding warts surfaced while writing these
scenarios and are worth knowing about when a scenario mysteriously
"clicks nothing" on TUI: `tui/split.rs` paints the divider at
`round(divider_bounds.x)` while `SplitLayout::hit_test` uses the
unrounded rect (the fix `tui/split_tree.rs` already got for #452), and
a dialog whose total height is an odd number of lines centres on a
half-line in an even-height viewport, so its painted button row and its
hit region land one cell apart.
## Backend testability requirement
Every backend MUST support headless paint-to-memory so tests don't
need a real display, terminal, window manager, or font server.
- TUI: `ratatui::Buffer` (in-memory char + style cells).
- GTK: `cairo::ImageSurface::create(Format::ARGB32, w, h)` + Pango
layout queries.
- macOS: `quadraui::macos::headless::BitmapSurface` (CGBitmapContext +
pixel readback, top-left origin matching `QuadraView`). Integrates
with `MacBackend::enter_frame_scope`; the full in-window rasteriser
surface (chrome / content / MSV / containers / overlays) drives the
same code paths as the live runner.
- Windows (when implemented): `ID2D1Bitmap` as offscreen render target.
New backends ship with their harness on day one. Once a backend has a
`ConformanceDriver` impl, it MUST also implement
[`FrameInventory`](#frameinventory-the-portable-paint-inventory-contract-quadraui490) —
see the backend checklist above.
## Backend `todo!()` gate (quadraui#923)
`tests/backend_todo_gate.rs` guards against a specific latent-abort shape:
an un-cfg-gated `todo!()`/`unimplemented!()` inside one of the four real
per-platform `Backend` impls (`TuiBackend`, `GtkBackend`, `MacBackend`,
`WinBackend`) that's reachable when that backend is compiled for its own
native target. That already shipped once —
`MacBackend::draw_minimap`/`minimap_layout`/`draw_image` had no `cfg` at
all and aborted vimcode's macOS GUI on first paint (vimcode#896, fixed in
`dbb3023`) — and a raw `todo!()` count can't tell that case apart from
Win-GUI's dozens of legitimate `cfg`-gated stubs (there purely so
`cargo check --features win` can type-check `WinBackend`'s trait
completeness on Linux). Counting without classifying would be red on
Win-GUI's stubs from day one, get muted, and miss the next macOS-shaped
case exactly as the absence of this test did before #923.
**Per-backend dangerous (un-gated, reachable-on-native) count, measured by
parsing each backend's source with `syn` rather than by line-adjacency**
(a `#[cfg(...)]` binds to exactly one following statement or item — see
the test's own module doc for the sibling-statement shape a proximity
check gets wrong):
| backend | dangerous (un-gated) `todo!()`/`unimplemented!()` |
|---|---|
| tui | 0 |
| gtk | 0 |
| macos | 0 |
| win | **59** (`src/win/backend.rs`, tracked as quadraui#924 — see that file's `Backend` impl; none in `src/win/run.rs`, whose 2 stubs are gated by a `#[cfg(not(target_os = "windows"))]` on the enclosing `pub fn`) |
Win-GUI's real total of `todo!()`/`unimplemented!()` call sites (ignoring
doc-comment mentions) is 81, not the "83 legitimate stubs" #923 was
originally filed against: 22 are genuinely excluded from a native Windows
build (2 via a `#[cfg(...)]` directly on the macro call — `begin_frame`/
`end_frame` — and 20 via a `#[cfg(...)]` on the whole enclosing `{ }` block,
e.g. `minimap_layout`/`list_layout`), and the other 59 carry no cfg that
excludes them and are reachable on Windows itself whenever `self.surface`
is `None`. Fixing those 59 is quadraui#924's job, not #923's — this test
ships a frozen baseline (`KNOWN_DANGEROUS_TODOS`) naming exactly those 59
call sites so it passes today without #924 having landed, while still
failing the instant a 60th un-gated stub appears anywhere in one of the
four backends.
**Recognised `cfg` spellings** (anything else is refused as
`Unclassifiable` — a hard test failure naming the exact attribute, never a
silent pass in either direction):
- `cfg(test)` — exempt everywhere (never compiled outside a test build).
- `cfg(not(test))` — not exempt (this is the shipped arm).
- `cfg(target_os = "X")` — exempt iff `X` differs from the file's own
backend's native `target_os` (`"macos"` for `src/macos/`, `"windows"`
for `src/win/`).
- `cfg(not(target_os = "X"))` — exempt iff `X` matches that native
`target_os`.
**Refused, not classified:** `cfg_attr(...)` in any form, `cfg(any(...))`,
`cfg(all(...))`, `cfg(not(any(...)))`/`cfg(not(all(...)))`, bare
`cfg(unix)`/`cfg(windows)`, a `target_os` comparison inside `src/gtk/` or
`src/tui/` (no native `target_os` is defined for those — they ship
cross-platform today), or a `cfg(...)` whose inner tokens don't parse as a
`syn::Meta` at all. Any of these appearing in a real backend fails the
test loudly rather than guessing which way it resolves.
**Scope:** only `src/tui/`, `src/gtk/`, `src/macos/`, `src/win/` — the
`impl Backend` blocks actually reachable from `quadraui::{tui,gtk,macos,win}::run`.
`src/testing/mod.rs`'s `RecordingBackend` (a `pub`, always-compiled
`Backend` mock for unit tests, never the backend an `AppLogic` is actually
run against) and `compose/*`'s private `#[cfg(test)] mod tests` mocks are
deliberately out of scope.
**Statement-position macros are covered too.** `syn` parses a
semicolon-terminated (or brace-delimited) macro invocation used as an
ordinary mid-block statement — `todo!("x"); more_code();` — as a distinct
`Stmt::Macro` node, not the tail-expression `Stmt::Expr(Expr::Macro(...))`
shape. The walker has a `visit_stmt_macro` override running the identical
classification logic for that shape, so an idiomatic non-tail `todo!()`
can't slip past the gate unclassified the way a `visit_expr`-only walker
would miss it; see `backend_todo_gate.rs`'s `self_test` module for the
pinned fixtures. (A per-match-arm `#[cfg(...)]` is the same class of gap in
principle — no `visit_arm` override — but no backend uses that pattern
today, so it's undetected-but-inert rather than a live blind spot.)
## Live-app headless smoke (GD-5, quadraui#450)
The offscreen `GtkDriver` above is deliberately display-free — no
`gtk::init`, no `Application`, no window, no `GdkDisplay` — which means
it structurally cannot catch bugs that only exist in a *real* window:
raw GDK signal delivery, widget realization/allocation, IME, or the real
OS clipboard. That's exactly the bug class that motivated this: #437
(`gtk_terminal` opening with a tiny/garbled window, paste not working at
all) only reproduced against a live window.
`quadraui::gtk::run` (every `gtk_*` example goes through it — no
per-example code needed) honours `QUADRAUI_GTK_SMOKE_MS` /
`QUADRAUI_GTK_SMOKE_PASTE` to run a scripted check against the real
window and exit 0/non-zero — see the "Headless smoke mode" section of
`quadraui/src/gtk/run.rs`'s module doc for exactly what's checked
(window/`DrawingArea` size floor, OS clipboard round-trip + a synthetic
Ctrl-V through the real interception path). `quadraui/scripts/gtk_smoke.sh`
wraps the `xvfb-run -a env GSK_RENDERER=cairo ...` invocation.
**This is an operator-run tier, not a CI gate.** The `gtk` CI job is
deliberately Xvfb-free (see the comment in `.github/workflows/ci.yml`) —
running it requires a box with `xvfb` installed (the quadraui#450
`gtk-headless` capability routing). Only the size/clipboard *assertion
logic* is unit-tested in-repo (`gtk::run::smoke_tests`, no display
required); the live launch itself needs a human or an operator-run box
to actually invoke `scripts/gtk_smoke.sh`.
## Win-GUI C4 playbook (native residue, operator-run, quadraui#746)
C4 ("Native residue — never shared" in the [tiers table above](#the-tiers-a-new-backends-burn-down-checklist))
is exact colours, font rendering, wide-glyph pixels and live-window
smoke — none of it assertable from shared code by design. This section
is the manual pass an operator runs on `dell64` (see CLAUDE.md's
"Win-GUI: building and testing for real"), plus the one-time question
that motivated writing it down.
### Step 1 (done, 2026-09-04): the Wine question, settled
The concern going in: `dell64`'s automated `cargo xwin test ...`
path runs the cross-compiled `.exe` through *something* other than a
human logging into Windows, so a green run there might be an artifact
of emulation rather than proof the rasteriser works — Direct2D/
DirectWrite support under a generic Linux binfmt+Wine setup is
partial, and a red *or* green result under Wine wouldn't say much
about real Windows either way.
**That premise doesn't hold on this box, and it's worth recording why
rather than re-asking the question per issue.** `dell64` has no Wine
installed at all:
```console
$ which wine wine64
wine not found
wine64 not found
```
The binfmt interpreter CLAUDE.md's trap #2 refers to
(`CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_RUNNER=env`, letting
`binfmt_misc` "do the exec directly") is **WSL2's own interop
handler**, not a userspace PE loader:
```console
$ cat /proc/sys/fs/binfmt_misc/WSLInterop
enabled
interpreter /init
flags: P
offset 0
magic 4d5a
```
`/init` hands the process to the real Windows host over the WSL
interop pipe — confirmed by shelling out to the actual Windows side
(`/mnt/c/Windows/System32/cmd.exe`, not a wrapper) and reading back
its own idea of itself:
```console
$ /mnt/c/Windows/System32/cmd.exe /c "ver & hostname"
Microsoft Windows [Version 10.0.26200.9168]
dell64
```
That's a real, installed Windows 11 build on real hardware named
`dell64` — the same host CLAUDE.md already names as "physically to
hand." **So the `cargo xwin test --target x86_64-pc-windows-msvc
-p quadraui --features win` / `CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_RUNNER=env`
path is not an emulator run to be distrusted — it already executes
the compiled binary against the genuine NT kernel and the genuine
Direct2D/DirectWrite stack.** A pass there (re-run today as this
issue's step 1, same command CLAUDE.md documents):
```console
$ RUSTFLAGS="-C target-feature=+crt-static" \
CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_RUNNER=env \
cargo xwin test --target x86_64-pc-windows-msvc -p quadraui --features win
...
test result: ok. 1760 passed; 0 failed; ... # lib unit tests
Running tests/conformance.rs
test result: ok. 39 passed; 0 failed; ... # C0 + C1, win burn-down column
Running tests/win_example_driver.rs
test result: ok. 5 passed; 0 failed; ...
Running tests/win_sxs_manifest.rs
test result: ok. 3 passed; 0 failed; ...
```
**Consequence: C0–C3 numbers gathered via the documented `dell64`
runner are real-Windows evidence already — they do not need a second,
separate "was that actually Windows" re-check.** What that run
*cannot* prove is anything requiring a visible window or an
interactive desktop: `HeadlessSurface` never creates an `HWND`, and a
`cargo xwin test`-launched process may run outside the interactive
window station even though it's genuine native code. That residual
gap — not "is it Wine", but "is there a window and a human looking at
it" — is exactly C4, and is what the checklist below covers.
**Doctest exit 53 — resolved (2026-09-07).** The aside that used to sit
here reported `cargo xwin test`'s **doctest** binaries failing with exit
53 — the "missing vcruntime" signature of CLAUDE.md's trap #1 — even
with `crt-static` set, and guessed that `rustdoc`'s harness doesn't
inherit `RUSTFLAGS`. That guess was right, and the fix is one more env
var: `rustdoc` compiles each doctest itself and reads **`RUSTDOCFLAGS`**,
not `RUSTFLAGS`. The full command, all three variables required:
```bash
RUSTFLAGS="-C target-feature=+crt-static" \
RUSTDOCFLAGS="-C target-feature=+crt-static" \
CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_RUNNER=env \
cargo xwin test --target x86_64-pc-windows-msvc -p quadraui --features win
```
`tools/win-test.sh` wraps exactly that line so none of the three has to be
retyped from memory:
```console
$ tools/win-test.sh
...
Doc-tests quadraui
test result: ok. 11 passed; 0 failed; 22 ignored; ...
```
**Run the doctest leg through `tools/win-test.sh`, not by retyping the
env-var prefix.** The Windows run needs three environment variables and
the doctest leg is the part that notices when one is missing — while
all 20 integration-test targets sail past. Worse, *two different*
missing variables produce the *same* headline number, `9 failed` out of
11 (the two survivors are the `compile` / `compile fail` doctests,
which are never executed), so the count alone cannot tell you which:
| Missing variable | What each failing doctest prints |
|---|---|
| `RUSTDOCFLAGS="-C target-feature=+crt-static"` | `Test executable failed (exit status: 53)` |
| `CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_RUNNER=env` | `Couldn't run the test: No such file or directory (os error 2)` — rustdoc got cargo-xwin's hard-coded `wine` as its `--runtool`, and there is no wine on this host |
Both were hit for real: the first by #832's own smoke run, which
reported "9 of 11 doctests failed" and sent a worker looking in the
diff. The wrapper exists so neither can happen again;
`tools/lint` (`quadraui-repo-lint`, #1110's meta-test-relocation half —
moved out of `quadraui/tests/` so an outside contributor's `cargo test`
never fails on it) asserts
`tools/win-test.sh` keeps setting all three variables and keeps running the
same cargo line spelled out above.
Note that a `.cargo/config.toml` would *not* have prevented the second
one — cargo-xwin injects the runner into cargo's environment, and env
beats config.
**A fifth trap, unrelated to the above: `<binary>.exe: Invalid argument`
is the Windows host's code-integrity policy, not a broken build.** A
freshly-linked, unsigned `.exe` is sometimes refused by Device Guard /
Smart App Control while its reputation is still being resolved;
`binfmt_misc` surfaces that refusal as `EINVAL`, so the shell prints
`…exe: Invalid argument` and cargo reports `test failed` for a test
binary that never ran a single test. Confirm it with
`/mnt/c/Windows/System32/cmd.exe /c "<C:\ path to a copy> --list"`, which
prints the real reason (`was blocked by your organization's Device Guard
policy`); diffing the PE headers against a binary that *does* run will
show nothing, because nothing is wrong with it. **Remedy: re-run the
command.** The block clears on its own — the identical bytes exec fine a
minute later. It is intermittent and picks different victims each run
(doctest binaries especially, since each is freshly linked into a new
`/tmp` dir), so treat a lone `Invalid argument` as a flake and only
investigate if the *same* target fails a re-run.
### Manual pass checklist
Run after any change touching `src/win/backend.rs`'s rasterisers, and
otherwise periodically as a spot-check. Needs an interactive `dell64`
desktop session (RDP or console) — `cargo xwin run`, not `xwin test`,
so a real `HWND` opens:
```sh
RUSTFLAGS="-C target-feature=+crt-static" \
cargo xwin run --target x86_64-pc-windows-msvc --example win_appshell_demo --features win
```
1. **Theme colours actually applied.** `WinBackend::set_theme` is
wired (#724), but individual rasterisers are migrated
piecemeal, and the *majority* still aren't. Run
`grep -rln "Theme::default()" src/win/*.rs` yourself before relying
on any specific file list here — it goes stale fast and a wrong
worked example is worse than none. As of `680d7ce`, that grep
returns ~38 of the ~40 files in `src/win/`, including `tab_bar.rs`
and `status_bar.rs` (whose module doc says outright that
`WinBackend` has no live-theme wiring yet) — **those are not
migrated**, don't use them as the "known good" reference. The
rasterisers `WinBackend::draw_*` genuinely calls with
`&self.current_theme` (check `backend.rs`, not the per-file grep,
since a file can define both a themed production path and an
unthemed `#[cfg(test)]` fixture that also calls `Theme::default()`)
include `draw_text_input`, `draw_scrollbar`, `draw_command_line`,
`draw_terminal`, `draw_text_display`, `draw_drop_overlay`,
`draw_board`, `draw_minimap`, `draw_pipeline_view`,
`draw_command_center`, and `draw_diff_view`. Confirm chrome painted
by one of *those* (e.g. the text input caret/selection colours, or
the scrollbar thumb) visibly reflects a non-default theme; confirm
everything else — including the tab bar and status bar — is a
*known* gap, not a surprise.
2. **Font rendering and hinting.** Compare label and editor text at
100% and 150% Windows display scaling (Settings → System → Display)
against the same text on the `tui`/`gtk` backends run side by side.
Look for: correct weight (no accidental bold/thin substitution),
no clipped descenders/ascenders, hinting that doesn't blur at
small sizes.
3. **Wide-glyph / CJK / emoji advance** (pairs with #703's shared
scale logic, `src/win/text.rs`). No dedicated `win_terminal`
example exists yet (`win`'s terminal rasteriser is
headless-tested only — `win::terminal::tests`), so use
`win_appshell_demo`'s editor pane: type or paste a line mixing
ASCII with CJK (`" 1: 日本語.rs "`, the same fixture
`tabbar.wide_label_text_fidelity` uses) and an emoji. Confirm each
wide glyph occupies two cell-widths' worth of advance with no
overlap into the next character and no stray gap — the exact
failure mode #703 fixed for `gtk`/`macos`/`tui`.
4. **Live-window resize / DPI-change behaviour.** Drag-resize the
window across a wide range (including down to something absurdly
small) and confirm chrome relayouts without artifacts or a crash.
Drag the window between two monitors at different scale factors (or
change scaling live via Settings while the window is focused) and
confirm text/chrome re-rasterises crisply at the new DPI rather than
staying blurry-scaled from the old one.
5. **Cursor shapes.** Hover each interactive zone (resize edges, text
inputs, buttons, splitters if the example under test has any) and
confirm the OS pointer glyph changes appropriately (I-beam over
text, resize arrows over splits/edges, hand/pointer over buttons)
and reverts on leave.
### Where to record results
Log a dated entry **in this section** (not a separate file — a later
reader should not have to hunt) each time the manual pass above is
run, so "verified natively" is distinguishable from "green in CI" at a
glance:
| Date | Rev | Run by | Result |
|---|---|---|---|
| 2026-09-04 | `9452d50` (branch tip before this doc commit — the code state actually exercised by the `cargo xwin test` run below) | operator (dell64) | Step 1 (Wine question) done — see above. Manual checklist (1–5) not yet run; no interactive `dell64` session available for this pass. |
CI's `windows-latest` leg and `cargo xwin test` on `dell64` both prove
"compiles and runs headless" — neither is a substitute for a row in
this table. Only a row here means a human actually watched the window.
## What unit tests don't cover
Animation cadence, font-rendering quirks across host platforms,
terminal-specific edge cases (kitty vs xterm vs urxvt), exact color
choices, accessibility heuristics, "does this feel right". These
remain manual smoke / human review. Goal: every story ratchets
harness coverage forward so the manual-residue surface shrinks
toward "things that genuinely need eyes".
- **Native message-dialog visibility** (`PlatformServices::show_message_dialog`,
quadraui#666). `GtkDriver` paints `cairo::ImageSurface` offscreen — it
never opens a real GTK `Application`/window, so it structurally cannot
see a `gtk4::AlertDialog`, which is a *separate* native window GTK owns
outside the `DrawingArea` this crate rasterises into. `scripts/gtk_smoke.sh`'s
generic live-window check (window/`DrawingArea` size floor, optional
clipboard round-trip — see "Live-app headless smoke" above) doesn't open
a dialog either, so it proves the window itself is alive, not that the
alert appears or that its buttons behave. Same structural gap as
`PlatformServices::show_file_open_dialog`/`show_file_save_dialog`/
`show_folder_open_dialog` (#427, #935), which has never had automated
coverage of the native picker actually appearing. Manual smoke, until a
`GtkDriver` successor can see native windows:
1. `cargo run --example gtk_message_dialog --features gtk`
2. Press `m`. Confirm a real OS dialog titled "Discard unsaved
changes?" appears, parented to (centered on) the demo window, with
"Keep Editing" right of "Discard" (GNOME HIG: cancel leftmost,
default rightmost).
3. Click each button in turn (re-running step 1 between clicks) and
confirm the status bar reports "Kept editing" / "Discarded"
matching the click. Press `m` again and dismiss via Escape or the
window's close box; confirm the status bar reports "Cancelled".
4. Since quadraui#965, TUI is **not** part of this manual-only gap
anymore: `show_message_dialog` drives a real in-canvas
`compose::MessageDialogController` through a nested draw-and-read
loop (see `tui::services`'s module doc), and the `TuiDriver` tests
in `tests/tui_example_driver.rs` cover the full round trip
headlessly — a real button resolves, Escape resolves the cancel
button, no hang. `cargo run --example tui_message_dialog --features
tui` and press `m` is still worth a manual sanity check (arrow/Tab
between buttons, Enter, Escape, confirm the status bar reports
"Kept editing"/"Discarded" matching the choice) but is no longer
the *only* coverage the way GTK's native-window gap above still is.
5. **Windows** (quadraui#744 — `show_message_dialog` is real here,
`native_dialogs: true`): `cargo run --example win_platform_services
--features win` on `dell64` (see this doc's "Win-GUI: building and
testing for real" note in the repo's `CLAUDE.md`) and press `m`.
Confirm a real `TaskDialogIndirect` alert titled "Save changes?"
appears with Save/Don't Save/Cancel buttons (Save bold/default).
Click Save and Don't Save in turn (re-pressing `m` between clicks)
and confirm stderr reports `chose WidgetId("save")` /
`chose WidgetId("dont_save")` matching the click. Press `m` again
and dismiss via Escape or the dialog's close box; confirm stderr
reports `chose WidgetId("cancel")` both times — that button is
marked `is_cancel`, so the dismissal gesture round-trips to it
rather than reporting "dismissed with no choice".
- **macOS tests still depend on the host having Menlo installed**
(quadraui#1110, second half — deliberately **not** done by the
`tools/lint` extraction above). `MacDriver::new` (`src/macos/testing.rs`)
installs `make_font("Menlo", 14.0)` as the default font before every
headless macOS test runs, mirroring what `macos::run` installs for a
real window, and roughly a dozen individual tests additionally call
`make_font("Menlo", …).expect("Menlo installed on every macOS host")`
directly (`macos/text.rs`, `macos/status_bar.rs`, `macos/minimap.rs`,
`macos/toolbar.rs`) — 82 `"Menlo"` string literals across 37 files in
`src/macos/` in total (`rg '"Menlo"' quadraui/src/macos | wc -l`), most
of them reached transitively through `MacDriver::new`. `#929`'s
`Backend::register_font_from_memory` (`macos::text::register_font_from_memory`)
exists and is exercised by its own unit tests, but nothing wires an
embedded test-font's bytes into `MacDriver::new` or the direct
`make_font("Menlo", …)` call sites yet — a fork/CI runner without
Menlo (any non-macOS-with-full-font-set host, or a locked-down macOS
image) still fails the whole `--features macos` test leg on font
lookup, not on anything the PR under test actually changed. Doing this
right means picking (or generating) an embeddable monospace font,
re-deriving every pixel-metric assertion tuned to Menlo's actual glyph
advances against the embedded font's metrics instead, and re-verifying
`cargo test --features macos` on a real macOS host — large enough that
it needs its own tracked issue and its own PR rather than riding along
with the `tools/lint` extraction (which only moves *maintainer-workflow*
meta-tests, and touches nothing under `src/macos/`). Today this is
mitigated, not solved: `.github/workflows/macos.yml` only runs on
`macos-latest`, which ships Menlo, so CI itself doesn't hit this gap —
it is a real gap for anyone running `cargo test --features macos` on a
macOS host that lacks it.