Struct OneharnessProvider 

pub struct OneharnessProvider { /* private fields */ }

The default Provider: runs each prompt on a harness through the oneharness CLI (targets v0.3.7+ — the release carrying the mock/spy seam: run --mock-rules/--spy-file and the oneharness mock responder).

Wires five real oneharness features:

• --system <skill instructions> — the skill becomes a real system prompt on the underlying harness (e.g. --append-system-prompt for claude-code), instead of being inlined into the user message.
• --resume <session> — multi-turn respond calls thread the previous session_id so the harness sees a continuing conversation (and keeps its tool state, files, etc.) instead of being re-prompted with a stringified transcript. Used only for harnesses that report supports_resume in the registry (claude-code, opencode, cursor today); other harnesses fall back to the inline-transcript path.
• --events — normalized tool events ({kind, name, input, output, index}) lifted from each harness’s transcript, so consumers can analyze what the skill did, not just its final text. Attached to the assistant turn.
• Normalized usage (input_tokens, output_tokens, cost_usd) — surfaced on every turn so cross-model cost reporting is portable.
• Normalized failure_kind (auth, rate_limit, model_not_found, …) — classified provider errors so the CLI can distinguish a broken environment from a broken skill.

Note on approval mode: skilltest deliberately passes no --mode flag, so oneharness applies its own default (v0.3.0+ normalized approval modes). Users who need a different mode — e.g. bypass to let the skill take every action without prompting — configure it through oneharness’s own config (ONEHARNESS_MODE / its config file), keeping approval policy in one place.

Evals and the simulated user always run on the configured judge_harness, independent of the harness under test, so the evaluator does not drift with the matrix.

Implementations

impl OneharnessProvider

pub fn new(config: &OneharnessConfig) -> Self

Build a provider from its configuration.

Trait Implementations

impl Provider for OneharnessProvider

fn respond( &self, platform: &str, model: &str, skill: &SkillRef<'_>, messages: &[Message], session: Option<&str>, ) -> Result<AssistantTurn>

Run one assistant/skill turn given the conversation so far. session, when Some, is a handle returned by a previous respond call on this run that the provider may use to continue the same harness session (e.g. via oneharness run --resume); providers that don’t support continuation should ignore it.

fn respond_with_mocks( &self, platform: &str, model: &str, skill: &SkillRef<'_>, messages: &[Message], session: Option<&str>, mocks: Option<&MockPlan<'_>>, ) -> Result<AssistantTurn>

Like Provider::respond, but with a tool mock/spy plan: the provider must enforce the plan’s compiled ruleset on the turn’s tool calls and return the observed-call records on the turn (mock_calls).

fn respond_streaming( &self, platform: &str, model: &str, skill: &SkillRef<'_>, messages: &[Message], session: Option<&str>, on_event: &mut dyn FnMut(&ToolEvent) -> ControlFlow<()>, ) -> Result<AssistantTurn>

Like Provider::respond, but delivers each normalized tool event to on_event as it is observed, so a caller can stream events live and short-circuit. on_event returns ControlFlow::Break to abort the turn — the provider tears down the harness and returns the partial turn.

fn respond_streaming_with_mocks( &self, platform: &str, model: &str, skill: &SkillRef<'_>, messages: &[Message], session: Option<&str>, mocks: Option<&MockPlan<'_>>, on_event: &mut dyn FnMut(&ToolEvent) -> ControlFlow<()>, ) -> Result<AssistantTurn>

Like Provider::respond_streaming, with a tool mock/spy plan. Same contract as Provider::respond_with_mocks: the default supports no mocking and errs loudly on a present plan.

fn simulate_user( &self, model: &str, persona: &str, messages: &[Message], ) -> Result<UserTurn>

Produce one simulated-user turn.

fn judge( &self, model: &str, query: &JudgeQuery<'_>, messages: &[Message], ) -> Result<JudgeVerdict>

Score a criterion against the conversation.

fn supports_resume(&self, platform: &str) -> bool

True iff respond on platform will faithfully continue a prior session when given its session_id. The default is false; providers that support resume override this so the runner knows to thread the session id through.