eval-flake-rs 0.1.0

Detect flaky LLM eval cases by tracking pass/fail across repeated runs. Returns per-case flip-rate and an overall flakiness score. Zero deps.
Documentation
  • Coverage
  • 100%
    12 out of 12 items documented1 out of 4 items with examples
  • Size
  • Source code size: 19.52 kB This is the summed size of all the files inside the crates.io package for this release.
  • Documentation size: 369.65 kB This is the summed size of all files generated by rustdoc for all configured targets
  • Ø build duration
  • this release: 15s Average build duration of successful builds.
  • all releases: 15s Average build duration of successful builds in releases after 2024-10-23.
  • Links
  • Homepage
  • MukundaKatta/eval-flake-rs
    0 0 0
  • crates.io
  • Dependencies
    • —
  • Versions
  • Owners
  • MukundaKatta

eval-flake-rs

crates.io

Detect flaky LLM eval cases across repeated runs. Per-case classification (AlwaysPass/AlwaysFail/Flaky) plus a flip-rate score.

use eval_flake_rs::detect;
let runs = vec![
    vec![("c", true)],
    vec![("c", false)],
    vec![("c", true)],
];
let r = detect(&runs);

Zero deps. MIT or Apache-2.0.