Expand description
The provider boundary. skilltest never talks to a model directly; a
Provider runs the skill, plays the simulated user, and judges the
transcript.
There are two real implementations. OneharnessProvider (the default) runs
each prompt on a harness through the
oneharness CLI and parses
its JSON. CommandProvider speaks a small JSON-lines protocol (see
docs/protocol.md) and backs both the deterministic skilltest-fake-provider
used by the gate and any custom provider you write. The Provider trait
also lets the runner be unit-tested against an in-memory fake.
Structs§
- Assistant
Turn - An assistant/skill turn produced by the provider.
- Command
Provider - A
Providerbacked by an external command speaking the JSON protocol. - Judge
Query - A judge query: the criterion, its kind, and (for numeric) the scale.
- Judge
Verdict - A judge verdict: the raw value (bool or number) plus the stated reason.
- Oneharness
Provider - The default
Provider: runs each prompt on a harness through theoneharnessCLI. - Skill
Ref - A borrowed view of the skill under test, as sent to the provider.
- Usage
- Token / cost usage for one provider call.
- User
Turn - A simulated-user turn produced by the provider.
Enums§
- Judge
Kind - The kind of judgement requested.
Traits§
- Provider
- The provider boundary.
Functions§
- supports_
resume - The harnesses oneharness’s adapter table marks
supports_resume = true(claude-code’s--resume, opencode’s--session, cursor’s--resume). Kept in sync with theoneharness listregistry — when a new harness ships session continuation, add it here so the runner threadssession_id.