Skip to main content

Module regularized

Module regularized 

Source
Expand description

Diagnostics for regularized linear regression — the family the OLS diagnostics deliberately excluded, now provided as first-class types.

Regularization changes the fit in ways the OLS formulas can’t be reused for. Under ridge the hat matrix becomes H_λ = X(XᵀX + λI)⁻¹Xᵀ, so leverage, degrees of freedom, and everything built on them differ; lasso has no closed-form hat matrix at all. Each estimator therefore gets its own fitted type with the diagnostics that are actually well-defined for it:

  • RidgeFit — closed-form ridge (via SVD), with effective degrees of freedom Σ dⱼ²/(dⱼ²+λ), ridge leverage, GCV, effective AIC/BIC, and a ridge_vif that generalizes the OLS VIF and reduces to it at λ = 0 — the “VIF before/after regularization” comparison.
  • LassoFit — coordinate-descent lasso, whose natural degrees-of-freedom estimate is simply the size of the active set (Zou–Hastie–Tibshirani).

§Penalty conventions (read before comparing λ across estimators)

  • Ridge penalizes the centered predictors on their given scale; the intercept (a detected constant column) is never penalized. Ridge is not scale-invariant, so standardizing predictors first is the usual practice.
  • Lasso standardizes predictors internally and minimizes (1/2n)‖y − Xβ‖² + λ‖β‖₁, so its λ is on a different scale than ridge’s.

Neither is a drop-in for the other’s λ; they are documented per-type.

Structs§

LassoFit
A fitted lasso-regression model and its diagnostics.
RidgeFit
A fitted ridge-regression model and its diagnostics.

Functions§

select_lambda_gcv
Select the ridge penalty that minimizes GCV over a grid of candidate λs, returning the best RidgeFit.