Struct EvalRunner 

pub struct EvalRunner { /* private fields */ }

Batch runner: holds pointwise, pairwise, and RAG evaluators.

Implementations

impl EvalRunner

pub fn new(evaluators: Vec<Box<dyn Evaluator>>) -> Self

Creates a batch runner (pointwise evaluators only).

pub fn with_run_id(self, run_id: impl Into<String>) -> Self

Pins the run id stamped into the report and handed to Predictor::begin_run.

Use this to correlate an evaluation run with an external trace/CI record. Without it, each run() generates a fresh UUID v4.

pub fn with_pairwise(self, pairwise: Vec<Box<dyn PairwiseEvaluator>>) -> Self

Appends pairwise evaluators (P1-1, arena evaluation enters the unified report).

pub fn with_rag_evaluators(self, rag: Vec<Box<dyn RagEvaluator>>) -> Self

Appends RAG evaluators (B9: context precision/recall, answer relevancy), scored with each example’s retrieved contexts in rank order.

pub fn with_price_book(self, price_book: PriceBook) -> Self

Overrides the price book used for cost estimation (E1). Takes over the default set.

pub async fn run( &self, dataset: &Dataset, predictor: &dyn Predictor, ) -> Result<Report, EvalError>

Runs all evaluators on the dataset, returning the report.

P1-3: per-item tolerance — a failed predict records a "predict" failure and skips the example; a failed evaluator score records only that evaluator’s failure, others still score. P1-1: pairwise evaluators participate too, using (prediction, reference) as the A/B candidates (arena usage: put the answer under comparison in the reference slot).