Skip to main content

Module flake_detector

Module flake_detector 

Source
Expand description

Anytime-Valid Flake Detector (bd-1plj).

Detects flaky timing regressions in E2E tests without inflating false positives, using anytime-valid e-process statistics.

§Mathematical Model

For sub-Gaussian residuals r_t (mean 0 under null), the e-value at time t is:

e_t = exp(λ × r_t − (λ² × σ²) / 2)
E_t = ∏_{i=1}^t e_i

We reject H₀ (system is stable) when E_t > 1/α, providing anytime-valid Type I error control.

§Key Properties

  • Anytime-valid: Can stop testing early without invalid inference
  • No false positives in stable runs: E[E_t] ≤ 1 under H₀
  • Early detection: Strong evidence triggers early failure
  • Variable-length support: Works with different test run lengths

§Failure Modes

ConditionBehaviorRationale
σ = 0Clamp to σ_MINDivision by zero guard
E_t underflowClamp to E_MINPrevents permanent zero-lock
E_t overflowClamp to E_MAXNumerical stability
No observationsE_t = 1Identity element

§Example

use ftui_runtime::flake_detector::{FlakeDetector, FlakeConfig};

let mut detector = FlakeDetector::new(FlakeConfig::default());

// Observe latency deviations
let decision = detector.observe(latency_deviation);
if decision.is_flaky {
    eprintln!("Flaky test detected");
}

Structs§

EvidenceLog
Log entry for evidence tracking.
FlakeConfig
Configuration for the flake detector.
FlakeDecision
Decision returned by the flake detector.
FlakeDetector
Anytime-valid flake detector using e-process statistics.
FlakeSummary
Summary of flake detection run.