Skip to main content

run_ab

Function run_ab 

Source
pub fn run_ab(
    suite: &EvalSuite,
    suite_name: &str,
    runner: &dyn ModelRunner,
    cfg: &AbRunConfig,
) -> Result<AbReport>
Expand description

Runs every task in suite under both conditions through runner, scoring each answer, and assembles the paired report. The model is the only non-deterministic input.