test-lang 0.2.0

Compiler snapshot test harness for tokens, ASTs, and errors.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142

# test-lang v0.2.0 — The Core Harness

**The first release with working code.** v0.1.0 was scaffold — structure, tooling, quality gates, no logic. v0.2.0 is the harness itself: capture a stage's output as normalized text, assert it against an expected block, and get a line-level diff when they differ. Two public types you build (`Snapshot`, and `Diff`/`Change` describing what changed), one error you receive (`Mismatch`). No runtime dependencies; `no_std` + `alloc` supported.

## What is test-lang?

A snapshot test harness for language front-ends. Give it source, run that source through a lexer, parser, or diagnostics renderer, and assert the rendered result — the token stream, the syntax tree, the diagnostics — against a known-good block of text. When the output changes, you get a unified diff pointing at exactly what moved, and accepting the new behavior is a copy-paste. It owns no grammar and depends on no other front-end crate: it works over `core::fmt::Display` and `Debug`, so the same harness serves a hand-written lexer or a generated parser without coupling to either.

## What's new in 0.2.0

### `Snapshot` — capture, normalize, check

A `Snapshot` is the rendered output of some stage, held as normalized text. Four constructors cover the three things a front-end produces:

```rust
use test_lang::Snapshot;

// A token stream — one token per line, so the diff points at the exact token.
let tokens = Snapshot::per_line(["Ident(let)", "Ident(x)", "Eq", "Int(1)"]);

// A syntax tree — the pretty-printed `Debug` form.
#[derive(Debug)]
struct Binary { op: char, lhs: i64, rhs: i64 }
let tree = Snapshot::debug(&Binary { op: '+', lhs: 1, rhs: 2 });

// A rendered diagnostic — captured verbatim.
let diag = Snapshot::new("error: unexpected token\n  --> 1:5");
```

`check` compares against an expected block and returns a `Result` — no panic in the library, so the test author decides whether to `unwrap`, `expect`, or propagate with `?`:

```rust
# use test_lang::Snapshot;
let snapshot = Snapshot::per_line(["let", "x", "=", "1"]);
snapshot.check("let\nx\n=\n1").expect("token stream matches");
```

### Cross-platform normalization

The reason a snapshot written by hand on Linux matches output captured on Windows: both sides are normalized before comparison. CRLF and lone CR collapse to LF, trailing whitespace is stripped from every line, and trailing blank lines are trimmed. Interior blank lines and leading indentation — which are significant in a pretty-printed tree — are preserved.

```rust
use test_lang::Snapshot;

// Output captured on Windows: CRLF endings, a formatter's trailing space.
let captured = "error: bad token  \r\n  --> 1:1\r\n";
Snapshot::new(captured).check("error: bad token\n  --> 1:1").unwrap();
```

### `Diff` / `Change` / `Mismatch` — the failure report

A failed `check` returns a `Mismatch` whose `Display` is a unified diff — `-` lines expected, `+` lines produced — and whose `diff()` accessor exposes the `Diff` for programmatic inspection:

```rust
use test_lang::{Change, Snapshot};

let err = Snapshot::per_line(["let", "y", "=", "1"])
    .check("let\nx\n=\n1")
    .unwrap_err();

assert!(err.to_string().contains("-x")); // expected `x`
assert!(err.to_string().contains("+y")); // produced `y`

let edits = err.diff().changes().filter(|(c, _)| *c != Change::Equal).count();
assert_eq!(edits, 2);
```

The diff engine is a longest-common-subsequence edit script with a common prefix/suffix fast path: identical leading and trailing lines are matched in linear time before the quadratic LCS ever runs, so a near-miss snapshot only pays for the region that actually changed, and the reported diff stays tight instead of re-aligning lines that never moved.

### Property tests for the two invariants

`tests/proptests.rs` exercises the guarantees the harness rests on over a wide input space:

- A snapshot always checks clean against its own captured text.
- Normalization is idempotent — re-wrapping a snapshot's text is a no-op.
- `check` succeeds exactly when the two texts normalize equal.
- The diff is a faithful edit script: its `+`/` ` lines reconstruct the actual text, its `-`/` ` lines reconstruct the expected text.
- A diff is empty if and only if the two normalized inputs are equal.
- `per_line` equals a newline join of the rendered items.

### Design decision — no concrete front-end dependencies

The v0.1.0 roadmap planned to wire the `lexer`, `ast`, `parser`, and `diag` crates into the harness "when first used." They are not used, and that is deliberate: operating over `Display`/`Debug` snapshots any front-end's output without a compile-time dependency on the crate that produced it. Taking concrete dependencies would couple test-lang's release cadence to four other crates for no functional gain, and would violate the REPS rule that cross-crate coupling flow through abstractions rather than concrete internals. This is recorded in `dev/ROADMAP.md` per the anti-deferral rule as a dropped task, not a deferred one.

## Breaking changes

**None** — v0.1.0 exported no functional API. The unused `serde` feature and optional dependency from the scaffold were removed, and the scaffold's malformed `Cargo.toml` metadata (unquoted `keywords` / `categories`) was fixed.

## Performance

Local Criterion means (`cargo bench`, Windows x86_64, Rust stable). The path that matters in a test suite is the check on the matching side — the case a green suite runs on every pass:

- **Capture** (`per_line`, 16 lines): ~0.5 µs
- **Check, matching** (16 lines): ~1.2 µs
- **Check, matching** (256 lines): ~19 µs
- **Capture** (4096 lines): ~78 µs

## Verification

Run on Windows x86_64 (Rust stable 1.95.x) and on Linux via WSL2 Ubuntu; identical commands pass under the CI matrix (Linux / macOS / Windows × stable / 1.85):

```bash
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo clippy --all-targets --all-features -- -D warnings
cargo test
cargo test --all-features
cargo test --no-default-features
cargo build --examples
cargo +1.85 build --all-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo deny check
```

All green. Test counts at this tag: 30 unit + 8 integration + 6 property + 18 doctests.

## What's next

- **1.0.0 — API freeze.** `docs/API.md` marked stable, the SemVer promise recorded, and the full test + benchmark suite green on all three platforms.

## Installation

```toml
[dependencies]
test-lang = "0.2"

# no_std + alloc
test-lang = { version = "0.2", default-features = false }
```

MSRV: Rust 1.85.

## Documentation

- [README](https://github.com/jamesgober/test-lang/blob/main/README.md)
- [API Reference](https://github.com/jamesgober/test-lang/blob/main/docs/API.md)
- [CHANGELOG](https://github.com/jamesgober/test-lang/blob/main/CHANGELOG.md)

---

**Full diff:** [`v0.1.0...v0.2.0`](https://github.com/jamesgober/test-lang/compare/v0.1.0...v0.2.0).
**Changelog:** [`CHANGELOG.md`](https://github.com/jamesgober/test-lang/blob/main/CHANGELOG.md#020---2026-07-01).