rightkit-qa 0.2.0

Rust QA harness for Right Suite apps: engine black-box scenarios, hidden native UI driving through rightkit-control, Rust-test scenario wrapper, dynamic paid-provider mocks, loudness/WAV/PNG/frame validators, run lock, orphan sweep, hashed evidence.
Documentation

rightkit-qa

Rust QA harness for Right Suite applications. It provides declarative scenarios, engine and app control, run locking, isolated profiles, process handling, and hashed evidence.

The browser feature enables Chrome targets through rightkit-browser.

Rust integration tests

harness::Harness::scenario(name, tier, requires, |sc| ...) wraps a #[test] body with tier gating, requires-skip, a per-scenario evidence directory (<evidence root>/<slug>/evidence.json, hashed artifacts), an orphan-process check and a summary.jsonl skip ledger. A skipped scenario prints [qa] SKIP name: reason, because libtest reports it as ok.

Settings under rightkit cargo

rightkit cargo test starts the test binary with a scrubbed environment and refuses nested managed builds. Do not pass knobs as environment variables. Use this pattern instead:

  1. Declare tiers as cargo features of the test crate (full = [], paid = []), and pass them with Harness::with_feature_tiers(&[("full", cfg!(feature = "full")), ("paid", cfg!(feature = "paid"))]). Run them with rightkit cargo test -p my-e2e --features full,paid. The fast tier always runs.
  2. Put knobs in a KEY=VALUE settings file and read them through settings::Settings (get, require, get_bool, get_u64, get_f64, get_list, get_path). requires = ["env:KEY"] is answered from the same file.
  3. Build binaries under test first with a separate rightkit cargo build -p <bin>, then locate them next to the test executable's deps/ directory.

Search order (the first existing file wins; files are not merged):

  1. RIGHTKIT_QA_SETTINGS (a path; an error if it is set and the file is missing)
  2. <root>/.cache/qa/settings.env
  3. <root>/.cache/e2e/settings.env
  4. <root>/rightkit-qa.settings.env

A non-empty process environment variable overrides the file for a single key. This only applies to direct, unmanaged runs. The harness reads RIGHTKIT_QA_TIER to override the tier selection (Harness::with_tier_key renames it) and RIGHTKIT_QA_EVIDENCE to set the evidence root.

Mocks and validators

  • mock::MockServer::start_with(|req, base| Reply) serves dynamic, stateful and binary (Reply::bytes) responses for paid-provider contracts. MockServer::start(routes) keeps the static JSON routes.
  • loudness: pure-Rust ITU-R BS.1770-4 integrated loudness, EBU loudness range, sample peak and true peak. It is checked against ffmpeg ebur128 to within 0.2 LU.
  • media: WAV decode and stats, PNG non-trivial (pure Rust), and decoded-frame non-uniformity. ffmpeg is only needed to decode compressed audio and video, and its binary path is always passed in.

The browser target reaches a local dev server only through browser::allow_loopback_origin / BrowserTarget::launch_for_origins. That allowance is scoped to one scheme, host and port.

Publish through the RightKit release workflow.