Struct EvalRunner 

pub struct EvalRunner { /* private fields */ }

Evaluation runner — executes a suite against an agent using async metrics.

Implementations

impl EvalRunner

pub fn new() -> Self

Create a new EvalRunner with no metrics and default threshold 0.7.

pub fn metric(self, metric: Box<dyn AsyncMetric>) -> Self

Add a metric to score agent outputs with.

pub fn threshold(self, threshold: f64) -> Self

Set the minimum score threshold for a test case to pass.

A case passes if all metric scores are ≥ threshold.

pub async fn run( &self, agent: &dyn EvalAgent, suite: &EvalSuite, ) -> Result<EvalReport>

Execute the evaluation suite against the agent.

For each test case: calls agent.respond(input), scores with all metrics, marks passed/failed, and aggregates into an EvalReport.

Errors

Returns an error if the agent fails on any test case.

Trait Implementations

impl Default for EvalRunner

fn default() -> Self

Returns the “default value” for a type.