Expand description
Anytime-Valid Flake Detector (bd-1plj).
Detects flaky timing regressions in E2E tests without inflating false positives, using anytime-valid e-process statistics.
§Mathematical Model
For sub-Gaussian residuals r_t (mean 0 under null), the e-value at time t is:
e_t = exp(λ × r_t − (λ² × σ²) / 2)
E_t = ∏_{i=1}^t e_iWe reject H₀ (system is stable) when E_t > 1/α, providing anytime-valid
Type I error control.
§Key Properties
- Anytime-valid: Can stop testing early without invalid inference
- No false positives in stable runs: E[E_t] ≤ 1 under H₀
- Early detection: Strong evidence triggers early failure
- Variable-length support: Works with different test run lengths
§Failure Modes
| Condition | Behavior | Rationale |
|---|---|---|
| σ = 0 | Clamp to σ_MIN | Division by zero guard |
| E_t underflow | Clamp to E_MIN | Prevents permanent zero-lock |
| E_t overflow | Clamp to E_MAX | Numerical stability |
| No observations | E_t = 1 | Identity element |
§Example
ⓘ
use ftui_runtime::flake_detector::{FlakeDetector, FlakeConfig};
let mut detector = FlakeDetector::new(FlakeConfig::default());
// Observe latency deviations
let decision = detector.observe(latency_deviation);
if decision.is_flaky {
eprintln!("Flaky test detected");
}Structs§
- Evidence
Log - Log entry for evidence tracking.
- Flake
Config - Configuration for the flake detector.
- Flake
Decision - Decision returned by the flake detector.
- Flake
Detector - Anytime-valid flake detector using e-process statistics.
- Flake
Summary - Summary of flake detection run.