# Testing strategy
ComposeLens tests protect observable behavior: source fidelity, Compose processing, diagnostics,
redaction, deterministic output, and evidence boundaries. Coverage percentages are regression
alarms, not substitutes for meaningful assertions.
## Test layers
| Unit tests | Local parsing, classification, validation, and rendering rules |
| Syntax and round-trip | Authored bytes, spans, recovery, and deterministic reparsing |
| Typed model | Source-aware Compose forms and malformed partial results |
| Processing | Loading, interpolation, merge, includes, profiles, paths, references, and defaults |
| Generated output | Exact bytes, invalid input rejection, sensitivity, and parse-back |
| Preservation editing | Atomic span edits and byte-identical unrelated source |
| Compatibility | Exact implementation versions, evidence scope, and findings |
| Real-world corpus | Interactions from licensed deployment examples |
| Public API | External-consumer compilation and supported behavior |
| Repository policy | Supply chain, fixtures, documentation, schema, and workflow invariants |
| Application contracts | Independent real-application semantics and retained evidence |
Executable integration-test entry points and their focused responsibilities are indexed in
[`tests/README.md`](../tests/README.md). Do not copy every case into narrative documentation; test
names, fixtures, and assertions are the detailed source of truth.
## Expectations for changes
A behavior change normally adds one successful case and one relevant rejection or recovery case.
Also cover the boundary affected by the change:
- merge-sensitive behavior includes replacement, reset, override, and provenance;
- public API changes include an external-consumer test;
- generated output includes exact bytes and typed parse-back;
- sensitive values include redacted `Debug` and diagnostic assertions;
- parser changes include malformed recovery and byte preservation; and
- compatibility claims include version-scoped evidence or remain explicitly unknown.
Use the smallest fixture that isolates a rule. Add a real-world fixture only when it protects a
distinct interaction or known regression. Empty test binaries and coverage-only assertions are not
useful evidence.
## Deterministic pull-request checks
Run the complete repository gate from the Dev Container:
```console
./scripts/check-all.sh
```
It formats owned files and runs repository policy, all Rust targets, Clippy, tests, doctests,
Rustdoc, conformance contracts, real-world cases, public API checks, package verification, coverage
ratchets, MSRV checks, dependency policy, offline links, and SemVer analysis. Any edit after a
successful complete run requires another run.
Useful focused commands are:
```console
cargo ci-policy
cargo ci-test
cargo test --locked --test typed_model
cargo test --locked --test processing
cargo test --locked --test generated_rendering
cargo test --locked --test public_api
cargo test --locked --test real_world
cargo ci-application
./scripts/check-files.sh --check
```
The full gate remains mandatory before committing and publishing a pull request. PR CI uses a
trusted-base, fail-closed change plan: only plain public prose can select the smaller repository
file/link job plus the lockfile-age guard. Inline code, fenced or indented examples, non-public documentation,
code, policy, unknown, or mixed changes select all jobs. There is no independent executable-guide
suite, so those changes never use a reduced PR plan. Initial rollout against a base without the
planner also runs all jobs. Main pushes, manual CI, and the reusable release CI always run all jobs
for the exact tested commit. The stable `PR gate` accepts only planned skips and succeeds only when
every selected job succeeds.
The VS Code “Plan changed validation” task previews a local plan. “Validate changed files” runs
it against staged, unstaged, untracked, and committed branch changes, checking that the local
snapshot stays unchanged during validation. “Validate public prose only” rejects other profiles;
“Validate full (check-only)” explicitly runs the complete gate. Worktree-local build artifacts are
required to avoid stale path-bearing binaries from deleted worktrees. The complete gate rejects an
external `CARGO_TARGET_DIR` and its mock regression covers worktree relocation. These are iteration aids;
the complete pre-publication gate remains mandatory.
## Fixtures
Every fixture directory has a `fixture.toml` manifest describing its suite, provenance, files,
license, redistribution, modifications, secret review, environment assumptions, and expected
behavior. Relative paths must remain inside the fixture directory. External material needs an
immutable revision and redistribution permission; sensitive or identifying data must not be
committed.
The complete contract and real-world admission policy are in
[`fixtures/README.md`](../fixtures/README.md). Tests never read the process environment, access
referenced paths, contact registries, or start a runtime merely because a fixture mentions them.
## Real-application contracts
`cargo ci-application` runs the deterministic offline contracts in
`tests/application_conformance.rs`. Nextcloud and Forgejo inputs are pinned to immutable BoxFerry
revisions, but their expected native Compose meaning is asserted independently through ComposeLens
public APIs. The suite covers interactions across explicit interpolation, ordered loading, merge
tags and provenance, environment-file precedence and value states, profile and reference
processing, project boundaries, redaction, and generated-document parse-back.
These tests do not establish runtime compatibility and do not depend on BoxFerry code. Provider
configuration and externally supplied runtime observations are separately versioned under
[`conformance/`](../conformance/README.md). The local complete gate, full pull-request CI plans,
main CI, and release workflow invoke the application suite explicitly. A prose-only PR plan may
skip the Rust application suite, but the complete pre-publication gate and the exact-candidate
main/release gates prevent a skipped or empty contract from silently reducing release coverage.
Release validation invokes CI through its reusable workflow interface and a separate bounded native
provider-configuration matrix. The native matrix records the candidate SHA, run, and worker task,
executes only reviewed observed rows, and does not execute planned or runtime-effect work.
## Provider and runtime evidence
Ordinary Cargo tests do not execute external Compose providers or runtimes. Matrices, capture
commands, isolation requirements, reviewed records, and the distinction between planned and
observed runs live in [`conformance/README.md`](../conformance/README.md).
External evidence capture is explicit and ignored by default. A generated result becomes evidence
only after its versions, commands, checksums, environment, output, fixture identity, and absence of
sensitive data are reviewed and committed.
## Documentation and repository policy
Repository tests bound maintained narrative pages, keep the public website inventory stable, reject
obsolete duplicate ledgers, and verify the current release-line wording. Markdown formatting and
links are checked locally; scheduled automation performs networked external-link validation.
Authored YAML parser fixtures are excluded from generic formatting because malformed and byte-exact
input is part of their contract. Metadata and repository-owned complete YAML remain syntax-checked.