# Rust Testing Tasks
Add `bake-test` as a dependency of the private `bake/` package and regenerate its task links:
```sh
cargo bake --regenerate
cargo add --manifest-path bake/Cargo.toml bake-test
cargo bake --regenerate
```
The package registers `test`, `test:coverage`, and `test:external` tasks. `test` runs `cargo test --workspace --locked`; `test:external` runs Cargo tests in the selected downstream repositories without `--locked`, because the lockfiles need to resolve local path patches.
## Coverage
Install `cargo-llvm-cov` and run the Bake coverage task:
```sh
rustup component add llvm-tools-preview --toolchain stable
cargo +stable install cargo-llvm-cov --locked
rustup run stable cargo bake test:coverage
```
Run the task with the same toolchain that has `llvm-tools-preview`. If another Rust installation such as Homebrew's `cargo` comes first on `PATH`, plain `cargo bake` may use that compiler and fail to find the Rustup component.
The task calls the optional `test:before` hook once, runs documentation tests with Cargo, then runs workspace tests under `cargo-llvm-cov`. It reports each uncovered source region and fails if any measured region remains uncovered. Regions with identical source spans are merged across function instantiations; distinct source spans remain separate, including spans on the same line. The task covers the whole workspace by default; use `--package name` to limit coverage to one package. Coverage uses the default feature set unless `--all-features true` or one or more repeatable `--features name` arguments are supplied. Choose one feature configuration per invocation; the task rejects combining `--all-features` and `--features`.
### Invariant-only `unreachable!()` calls
Rust checks local type-level impossibilities, such as exhaustive matches over closed enums. It does not generally prove runtime invariants that depend on relationships between values, parser behavior, or upstream code. An `unreachable!()` is a runtime assertion that panics if that invariant is violated; it is not proof that the branch cannot occur.
Regions inside an `unreachable!()` invocation are excluded from coverage, including multiline invocations. This exclusion is automatic and does not need a marker. It applies only to Rust source regions contained within the macro call; other uncovered code on the same line remains measured. The source scan ignores strings and comments.
Coverage uses the source regions reported by LLVM without classifying regions by their source text. A reported region remains measured even if its span contains only a delimiter or a macro name: LLVM can map executable behavior to such a span, including a branch outcome mapped to a closing brace. Inspect an uncovered region and the behavior represented by its mapping; do not exclude it solely because its span looks syntactic.
If `cargo-llvm-cov` reports a genuinely invariant-only `unreachable!()` call, include its reason in the panic message:
```rust
_ => unreachable!("Only JSX events can be mismatched here")
```
The macro asserts that the path should not be reached; it does not prove that the invariant is correct. Use `unreachable!()` only after checking the invariant against supported inputs. Test valid inputs and the boundary that establishes the invariant; do not manufacture an impossible private state solely to execute the panic. All other source-backed regions must reach 100% coverage.
### Excluding files
Some code cannot run in CI, for example raw calls into a headset runtime, a window system, or audio hardware. As a last resort, a package can exclude specific files from the gate in its own `Cargo.toml`:
```toml
[[package.metadata.bake.test.coverage.exclude]]
path = "src/xr.rs"
reason = "Raw OpenXR runtime calls; needs a headset runtime."
```
Exclusion is not a convenience; the default answer to uncovered code is a test. Keep logic out of excluded files: put only the untestable calls in a dedicated file, keep it thin, and keep the decisions around those calls in covered modules.
- `path` names one existing file, relative to the package directory. It must stay inside the package, without `.` or `..` components. Glob patterns and directories are not supported, so each file carries its own reason and size, and a new file is measured until someone lists it.
- `reason` is required and must not be blank. Explain why the file cannot be tested in CI.
- The task reads the exclusions of every workspace member and fails before running tests if an entry has no reason, names a file that does not exist, or lists the same file twice. A renamed or deleted file therefore cannot leave a stale entry behind.
The task still instruments excluded files. It removes their regions from the gate, then lists every excluded file with its reason and its size from the unfiltered report: its source regions, how many of them ran, and the lines they span. A file that is not compiled in the selected configuration, such as code for another target, is listed with no regions in this run. The listing appears whether the gate passes or fails:
```text
Region coverage passed: 52830/52830 measured source regions covered (excluded: 2 unreachable regions, 1 file)
Excluded files, not counted towards the gate:
crates/client-quest/src/xr.rs (1124 regions, 0 covered; 651 lines): Raw OpenXR runtime calls; needs a headset runtime.
```
Exclusions are package metadata; there is no workspace-wide list, so each entry sits next to the code it describes.
The canonical GitHub Actions workflow below installs the required Rust components and coverage tool. The task measures the runner's target and feature configuration. If a crate contains architecture-specific code, run the same gate on each supported architecture; each run checks the code compiled for its target.
The coverage task runs documentation tests but does not include them in the coverage report; LLVM doctest coverage is still unstable. It uses the JSON function-region data to enforce 100% of measured source regions. This task measures source regions, not LLVM's separate experimental branch-coverage metric.
## Before-test hook
The `test`, `test:coverage`, and `test:external` tasks call the optional project task `test:before` once before running tests. Use it for resources that need to be prepared for either local or downstream tests, such as downloading fixtures or generating assets:
```rust,ignore
#[bake::task(name = "test:before")]
fn before_test(context: &mut bake::Context) -> bake::Result<()> {
let status = context.command("make").arg("test-assets").status()?;
if status.success() {
Ok(())
} else {
Err(bake::Error::new(format!("test asset preparation failed: {status}")))
}
}
```
The hook is optional. If it is not registered, the test tasks continue without calling it. External checkout and patch preparation happens before the hook; the hook runs once before Cargo tests begin. If no downstream repositories are configured, `test:external` returns a message without running the hook.
## External repositories
List selected downstream repositories in the root `Cargo.toml`:
```toml
[[workspace.metadata.bake.test.external]]
repository = "https://github.com/socketry/downstream-project"
branch = "main"
```
For a single-package project, use `[[package.metadata.bake.test.external]]`. `branch` defaults to `main`. Add `name = "checkout-name"` when you want the local checkout directory to use a specific name or avoid a name collision. If the list is empty or absent, `test:external` reports that no downstream repositories are configured.
The task clones each repository into `external/<name>/` on first use. It keeps existing Git checkouts and does not fetch, reset, switch branches, or discard local edits. To update a checkout, run Git commands inside it. If the metadata repository no longer matches the checkout, choose a new `name` or move the old directory yourself.
The task adds this workspace's crates.io-publishable packages to the cloned repository's `[patch.crates-io]` table, using relative paths back to the local workspace. Existing unrelated manifest content is preserved, and the same patches are not inserted a second time. If a conflicting patch already exists, the task stops and asks you to resolve that entry. The patch remains in the checkout so you can enter it and run `cargo test --workspace` manually to investigate a failure. Add `/external/` to the consumer repository's `.gitignore`, as this repository does.
Cargo still checks the downstream dependency's version requirement. If the local workspace version does not satisfy it, update that requirement in the checkout before rerunning external tests. The task checks Cargo's resolved dependency graph and stops if a downstream dependency silently resolves to the registry version instead of the local patch.
## Canonical GitHub workflows
Use this `test.yml` for Socketry Rust repositories. It runs formatting, Clippy, documentation tests, and workspace coverage, checks the publishable packages with the declared minimum Rust version, and checks license headers. The coverage task invokes the optional `test:before` hook and requires 100% coverage of measured source regions. Passing `--all-targets true` includes examples and benchmarks in the coverage run. Do not add a separate `cargo bake test` step to this job; the coverage task runs the tests itself.
Filter both `push` and `pull_request` to `main`. Pull request updates then run through `pull_request`, without a duplicate run from the feature-branch push; merging to `main` still runs the workflow after the merge. Keep `workflow_dispatch` for manual runs.
```yaml
name: Test
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v7
- uses: actions-rust-lang/setup-rust-toolchain@v2
with:
components: clippy, llvm-tools-preview, rustfmt
# Invalidate caches containing the retired socketry-cargo-bake launcher.
cache-shared-key: coverage-bake-launcher-v2
- name: Install Bake launcher
run: cargo install bake --locked
- name: Install coverage tool
run: cargo install cargo-llvm-cov --locked
- run: cargo fmt --all -- --check
- run: cargo clippy --workspace --all-targets --locked -- -D warnings
- name: Run tests and require complete source-region coverage
run: cargo bake --locked test:coverage --all-targets true
minimum-rust:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v7
- name: Read the declared minimum Rust version
id: rust-version
run: |
version=$(sed -n 's/^rust-version = "\(.*\)"$/\1/p' Cargo.toml | head -n 1)
test -n "$version" || { echo "Cargo.toml declares no rust-version" >&2; exit 1; }
echo "version=$version" >> "$GITHUB_OUTPUT"
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ steps.rust-version.outputs.version }}
- name: Build the publishable packages with the declared minimum Rust version
run: |
packages=$(cargo metadata --no-deps --format-version 1 --locked | jq -r '.packages[] | select(.publish != []) | "--package=\(.name)"')
cargo check $packages --all-targets --locked
license:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v7
with:
# Copyright lines come from the full history of the commits under review, not of GitHub's merge commit.
ref: ${{ github.event.pull_request.head.sha || github.sha }}
fetch-depth: 0
- uses: actions-rust-lang/setup-rust-toolchain@v2
with:
cache-shared-key: license-bake-launcher-v1
- name: Install Bake launcher
run: cargo install bake --locked
- name: Check license files and Rust source headers
run: cargo bake --locked license:check
test-result:
if: always()
needs: [test, minimum-rust, license]
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Require successful tests and coverage
env:
JOB_RESULTS: ${{ toJSON(needs) }}
run: |
echo "$JOB_RESULTS" | jq -e 'all(.[]; .result == "success")'
```
The `minimum-rust` job verifies the minimum supported Rust version that the root `Cargo.toml` declares with `rust-version`, in `[package]` or `[workspace.package]`. It checks every publishable package with its tests, examples, and benchmarks, skipping private packages such as the `bake/` task package, which may use newer tools. Every publishable package is checked with that version, so give them the same declaration. Raise the declaration rather than removing the job when a dependency needs a newer version.
The `license` job runs `license:check` from `bake-license`, which `base-project` links into the task package. It fails, naming each file and the fix, when `license:update` would change `license.md`, the README License section, or the license header of any tracked Rust file, including test fixtures such as trybuild compile-fail cases. The Base project release hook runs `license:update`, so this check makes a missing or stale header fail the pull request that introduces it instead of changing sources, and moving the line numbers that compile-fail snapshots record, in a release. To fix a failure, run `cargo bake license:update`, regenerate any compile-fail snapshots whose line numbers moved (for trybuild, `TRYBUILD=overwrite cargo test`), and commit the result. Copyright lines come from the Git history, so the job checks out the pull request's head commit with its full history; the check refuses a shallow clone. A squash merge records one author at the merge date, so a pull request with several authors, merged in a later year, or touching a file changed on `main` since it branched can still leave stale headers; the job's run on the push to `main` reports them, and they are fixed the same way before the next release. A repository whose task package does not link `bake-license` 0.2.9 or later, which provides the task, omits this job.
Require `test-result` alongside the publishing workflow's `check` job in the main-branch ruleset. For a matrix or several required test jobs, list every required job in `needs`. The aggregate runs even when a prerequisite fails, and succeeds only when every listed job succeeds. Keep experimental or diagnostic jobs outside that required gate.
The standard `test` task remains useful for quick local runs without a coverage report. CI uses `test:coverage` so it enforces the organization-wide source-region coverage requirement.
External compatibility testing is optional. Add `.github/workflows/external.yml` only when the Cargo metadata list contains one or more selected downstream repositories:
Use the same event filters as `test.yml`: pull requests targeting `main` and pushes to `main`. This prevents duplicate downstream test runs for a pull request branch while still checking the merged default branch.
```yaml
name: External Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v7
- uses: actions-rust-lang/setup-rust-toolchain@v2
- name: Install Bake launcher
run: cargo install bake --locked
- name: Run downstream compatibility tests
run: cargo bake --locked test:external
```
The external task creates fresh checkouts in the workflow's temporary runner and applies the local patches before testing. Remove this workflow when the metadata list becomes empty.