Skip to main content

Module roundoff

Module roundoff 

Source
Expand description

One definition of “this residual is indistinguishable from zero”.

Two places in the engine have to decide whether a fitted mean reproduces its response exactly: the formula path’s deterministic-Gaussian dispatch, which PREDICTS the state from the shape of the request, and the solver, which MEASURES it after the dispersion has been estimated. They must decide it the same way — a fit that is exact on one route and merely near-exact on the other reports a different scale, a different covariance, and a different criterion for the same data, which is how #2595 stayed invisible for a week.

The shared quantity is Wilkinson’s accumulated-roundoff growth factor. A floating-point sum of k operations carries a relative error bounded by

    γ_k = k·ε / (1 − k·ε),      ε = f64::EPSILON

so a linear predictor η_i = Σ_j x_ij β_j (+ offset) formed from p terms cannot be trusted below γ_{p+1} · (Σ_j |x_ij β_j| + |offset_i|) — and a residual y_i − η_i smaller than that is not evidence of misfit, it is the arithmetic. This is a derived bound, not a tuned threshold: it moves with the model width and the data scale and has no free parameter.

Functions§

roundoff_growth_factor
Wilkinson’s growth factor γ_k = k·ε/(1 − k·ε) for a sum of operations floating-point operations.
weighted_residual_is_at_roundoff_floor
Is a weighted residual sum of squares indistinguishable from zero?