Skip to main content

Module rolling_variance

Module rolling_variance 

Source
Expand description

Numerically stable O(1)-amortised rolling variance over a fixed window.

§Why this exists (quantwave-qkft)

The obvious sliding-window variance keeps Σx and Σx² and evaluates E[X²] - E[X]². That is O(1) per bar but catastrophically cancels when the window mean is large relative to the spread: both terms are ~mean² while the answer is ~σ², so the relative error is roughly eps * (mean/σ)². On raw prices that is harmless (~1e-11), but on a slowly-varying series such as log-price (lnP in 1..9 with per-bar moves of ~1e-4) the ratio is ~1e10 and the result drifts ~1e-6 relative away from a two-pass reference.

§The algorithm

Shifted-data variance with a deterministic periodic exact refresh:

  • We keep an origin K and accumulate Σ(x-K) and Σ(x-K)². Because K sits inside the current window, x-K is on the order of the window spread, not the window level, so the Σd² - (Σd)²/n subtraction cancels quantities of order n·σ² rather than n·mean². The mean-magnitude cancellation disappears entirely.
  • Add and remove are the exact algebraic inverses of each other, so a single evict/insert pair introduces no algorithmic drift beyond one rounding.
  • Rounding drift and origin staleness (a trending series eventually walks away from K) are bounded by recomputing origin, Σd and Σd² exactly from the window every period pushes. One O(period) pass per period bars is O(1) amortised, and the window has fully turned over since the previous refresh, so no rounding error ever survives a whole window.

§Determinism / batch-streaming parity

The refresh cadence is driven purely by a push counter, i.e. by bar index — never by data values, timers, or window content. Batch and streaming both drive this same accumulator through the same Next implementations, so the refresh lands on the same bars in both paths and the outputs stay bit-identical. There is no non-deterministic “recompute when error looks large” heuristic here, deliberately.

§Tradeoff

Per-bar cost is O(1) amortised but not O(1) worst case: one bar in every period does an O(period) refresh. A truly O(1)-worst-case removal-based rolling variance cannot bound drift indefinitely, so bounded periodic recomputation is the intentional price of correctness.

Structs§

RollingVariance
Sliding-window accumulator for population mean/variance.