Expand description
Fast collapse detection for connection goodput.
§Why this exists
The scheduler cannot repair a divergence it has not observed. Measurement of the earlier EWMA-only estimator showed a fixed detection cost of roughly 0.25–0.9 s when a source’s rate collapsed by ~97%: the makespan ratio against the fluid oracle was 2.44× on a 12 MB object and only fell to 1.02× at 192 MB, because the excess is a constant that amortises rather than a per-byte inefficiency. An EWMA is a smoother; asking it to detect a step change is asking the wrong question of it, and the lag is structural: after a collapse, the very samples the estimator needs arrive at the collapsed rate.
§What this does instead
Two independent mechanisms, because they fail in different directions.
- Dual-window ratio. A short window (recent arrivals) against a long window (the connection’s established rate). Responds within the short window, but is noisy.
- Two-sided CUSUM. Accumulates normalised deviations from the established
rate and fires when the running sum passes a threshold
h. Slower than the ratio test on a hard collapse but far more resistant to variance, so it catches slow degradation the ratio test smooths over.
A connection is graded on the strongest evidence available, and the grade —
not a raw rate — is what the scheduler acts on. Repair may fire on Suspect
long before the stall timeout would expire.
§The false-positive cost is real and asymmetric
A missed collapse costs the transfer up to a stall timeout. A spurious
detection costs one delta per unnecessary repair, and on a high-RTT path
delta is hundreds of milliseconds. The thresholds here are therefore set to
tolerate ordinary jitter — the stable_noisy_connection_is_not_flagged test
pins that behaviour, and it is as load-bearing as the detection tests.
Structs§
- Collapse
Detector - Per-connection collapse detector.
Enums§
- Health
- How healthy a connection looks, on the evidence so far.
Constants§
- WARMUP
- Samples required before the detector will grade anything. Below this it has no reference level, and grading noise as collapse is worse than not grading.