Struct Runner 

pub struct Runner<'a> { /* private fields */ }

Runs test cases against a provider using a configuration.

Implementations

impl<'a> Runner<'a>

pub fn new(provider: &'a dyn Provider, config: &'a Config) -> Self

Build a runner.

pub fn run_all(&self, cases: &[TestCase]) -> Result<Report>

Run every supplied case across the full platform × model matrix and collect a Report.

Errors

Propagates the first crate::Error from loading a skill or a provider failure. Eval failures are not errors — they are recorded in the report.

pub fn run_all_streaming( &self, cases: &[TestCase], on_event: &mut dyn FnMut(&StreamEvent<'_>) -> ControlFlow<()>, ) -> Result<Report>

Like Runner::run_all, but drives each turn through Provider::respond_streaming and delivers each skill tool event to on_event the instant it is observed. on_event returns ControlFlow::Break to short-circuit: the current run is torn down (the provider kills the harness), no further runs start, and the partial Report built so far is returned.

Errors

As Runner::run_all.

pub fn run_case(&self, case: &TestCase) -> Result<Vec<CaseRun>>

Run a single case across the matrix.

Errors

As Runner::run_all.