Skip to main content

Module model_comparison

Module model_comparison 

Source
Expand description

Honest, calibrated model comparison computed from machinery already present at the fit optimum — exact smoothing-corrected conditional AIC and zero-refit ALO elpd with an influence diagnostic (issue #946).

Every consumer (the topology race, the SAE fit payload, the compare entry point) reads the same two channels:

  • Corrected conditional AIC. The conditional AIC −2·ℓ + 2·edf treats the smoothing parameters as known and is biased toward complexity exactly where users rely on it (random-effect-vs-null, is-a-wiggle-real). The Wood–Pya–Säfken (2016, JASA) correction replaces edf = tr(F) by τ = tr(F) + tr(X'WX · Σ_ρ), where Σ_ρ is the smoothing-parameter uncertainty covariance in coefficient space. gam carries Σ_ρ exactly (assembled from the IFT dβ̂/dρ and the exact outer Hessian at the fit optimum, retained on the fit as UnifiedFitResult::smoothing_correction), so the correction is the first exact instance of this estimator — not the approximation mgcv must use, and not the omission most software ships.

  • ALO elpd. Pointwise log predictive densities evaluated at the ALO-corrected leave-one-out predictions (no refits — the ALO solves reuse the fit’s factored Hessian). The summed elpd is exactly Σᵢ ℓ(yᵢ|η̃₋ᵢ). A Pareto tail fit of the cross-observation fitted-vs-ALO ratio distribution is reported only as an influence diagnostic; it is not draw-wise PSIS-LOO and does not alter the pointwise contributions.

Both channels are corroboration: they ride alongside the evidence headline a race already produces, never replacing it.

Structs§

AloElpd
ALO predictive-accuracy summary at zero refit cost.
ComparisonReport
Result of comparing two fits on the same response: the paired predictive difference with its standard error plus the corrected-AIC gap. Both differences are oriented a − b: positive delta_elpd favours a, negative delta_aic_corrected favours a.
CorrectedEdf
Effective-degrees-of-freedom pair: the conditional tr(F) and the Wood–Pya–Säfken correction that accounts for smoothing-parameter uncertainty.
ModelComparison
The full comparison payload reported alongside a fit’s evidence headline.

Functions§

alo_elpd
ALO elpd from ALO-corrected leave-one-out predictions.
alo_elpd_from_family
ALO elpd for an engine-level family: map the fitted and ALO leave-one-out linear predictors through the family inverse link, score both with the per-row log-likelihood kernel, and compute the ALO elpd plus influence diagnostic.
compare
Paired comparison of two fits. The predictive difference is paired row-by-row, so the two fits must have been computed on the same response in the same order; we refuse the paired difference when the observation counts disagree and surface only the AIC gap.
corrected_edf
Exact Wood–Pya–Säfken corrected effective degrees of freedom.
model_comparison_from_unified
Assemble the comparison payload for a fitted GLM/GAM from the fit result plus optional ALO diagnostics.