Skip to main content

multi_seed_eval

anno_eval::eval::sampling

Function multi_seed_eval 

Source 
pub fn multi_seed_eval<F>(eval_fn: F, seeds: &[u64]) -> (f64, f64, f64, f64)
where
    F: Fn(u64) -> f64,
Expand description

Run evaluation with multiple seeds and aggregate variance.

§Research Context

Single-seed evaluations are unreliable:

  • F1 can vary ±5% across seeds on small datasets
  • Always report mean ± CI, not point estimates

§Arguments

  • eval_fn - Evaluation function that takes a seed and returns F1 score
  • seeds - Seeds to run (recommend 5+)

§Returns

(mean, std_dev, min, max) of F1 scores