Struct EvalRunner 

pub struct EvalRunner { /* private fields */ }

Batch runner: holds both pointwise and pairwise evaluators.

Implementations

impl EvalRunner

pub fn new(evaluators: Vec<Box<dyn Evaluator>>) -> Self

Creates a batch runner (pointwise evaluators only).

pub fn with_pairwise(self, pairwise: Vec<Box<dyn PairwiseEvaluator>>) -> Self

Appends pairwise evaluators (P1-1, arena evaluation enters the unified report).

pub fn with_price_book(self, price_book: PriceBook) -> Self

Overrides the price book used for cost estimation (E1). Takes over the default set.

pub async fn run( &self, dataset: &Dataset, predictor: &dyn Predictor, ) -> Result<Report, EvalError>

Runs all evaluators on the dataset, returning the report.

P1-3: per-item tolerance — a failed predict records a "predict" failure and skips the example; a failed evaluator score records only that evaluator’s failure, others still score. P1-1: pairwise evaluators participate too, using (prediction, reference) as the A/B candidates (arena usage: put the answer under comparison in the reference slot).