Expand description
Numerically stable O(1)-amortised rolling variance over a fixed window.
§Why this exists (quantwave-qkft)
The obvious sliding-window variance keeps Σx and Σx² and evaluates
E[X²] - E[X]². That is O(1) per bar but catastrophically cancels when the
window mean is large relative to the spread: both terms are ~mean² while
the answer is ~σ², so the relative error is roughly eps * (mean/σ)².
On raw prices that is harmless (~1e-11), but on a slowly-varying series such
as log-price (lnP in 1..9 with per-bar moves of ~1e-4) the ratio is ~1e10
and the result drifts ~1e-6 relative away from a two-pass reference.
§The algorithm
Shifted-data variance with a deterministic periodic exact refresh:
- We keep an
originK and accumulateΣ(x-K)andΣ(x-K)². Because K sits inside the current window,x-Kis on the order of the window spread, not the window level, so theΣd² - (Σd)²/nsubtraction cancels quantities of ordern·σ²rather thann·mean². The mean-magnitude cancellation disappears entirely. - Add and remove are the exact algebraic inverses of each other, so a single evict/insert pair introduces no algorithmic drift beyond one rounding.
- Rounding drift and origin staleness (a trending series eventually walks
away from K) are bounded by recomputing
origin,ΣdandΣd²exactly from the window everyperiodpushes. One O(period) pass perperiodbars is O(1) amortised, and the window has fully turned over since the previous refresh, so no rounding error ever survives a whole window.
§Determinism / batch-streaming parity
The refresh cadence is driven purely by a push counter, i.e. by bar index —
never by data values, timers, or window content. Batch and streaming both
drive this same accumulator through the same Next implementations, so the
refresh lands on the same bars in both paths and the outputs stay
bit-identical. There is no non-deterministic “recompute when error looks
large” heuristic here, deliberately.
§Tradeoff
Per-bar cost is O(1) amortised but not O(1) worst case: one bar in every
period does an O(period) refresh. A truly O(1)-worst-case removal-based
rolling variance cannot bound drift indefinitely, so bounded periodic
recomputation is the intentional price of correctness.
Structs§
- Rolling
Variance - Sliding-window accumulator for population mean/variance.