eval-flake-rs 0.1.0

Detect flaky LLM eval cases by tracking pass/fail across repeated runs. Returns per-case flip-rate and an overall flakiness score. Zero deps.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

# eval-flake-rs

[![crates.io](https://img.shields.io/crates/v/eval-flake-rs.svg)](https://crates.io/crates/eval-flake-rs)

Detect flaky LLM eval cases across repeated runs. Per-case
classification (AlwaysPass/AlwaysFail/Flaky) plus a flip-rate score.

```rust
use eval_flake_rs::detect;
let runs = vec![
    vec![("c", true)],
    vec![("c", false)],
    vec![("c", true)],
];
let r = detect(&runs);
```

Zero deps. MIT or Apache-2.0.