Struct EvalSuite 

pub struct EvalSuite {
    pub id: String,
    pub cases: Vec<EvalCase>,
    pub scorer: Arc<dyn AsyncScorer>,
}

Fields

id: String

cases: Vec<EvalCase>

scorer: Arc<dyn AsyncScorer>

Scorer used to grade each case. The field is typed as Arc<dyn AsyncScorer> so judges that genuinely await (LLM judges, retrieval-grounded checks, network probes) compose directly without a block_on bridge. Sync Scorer impls are promoted into AsyncScorer via the blanket impl in crate::scorer, so callers can construct with Arc::new(MySyncScorer) as Arc<dyn AsyncScorer> (or rely on type inference at the field-init site).

Implementations

impl EvalSuite

pub async fn run(&self, callable: &dyn Callable) -> Result<EvalRun>