Skip to main content

pass_at_k

Function pass_at_k 

Source
pub fn pass_at_k(n: usize, c: usize, k: usize) -> f64
Expand description

Compute pass@k: unbiased estimator of functional correctness.

Formula: 1 - C(n-c, k) / C(n, k)

where n = total samples, c = correct samples, k = top-k threshold.

Returns a value in [0, 1] where 1.0 means all k samples pass.

§Arguments

  • n - Total number of generated code samples
  • c - Number of correct (passing) samples
  • k - Number of samples to consider (typically 1, 10, or 100)

§Edge Cases

  • If k > n, returns if c > 0 { 1.0 } else { 0.0 }
  • If c >= n, returns 1.0
  • If c == 0, returns 0.0