regression_diagnostics/regularized/
mod.rs

1//! Diagnostics for **regularized** linear regression — the family the OLS
2//! diagnostics deliberately excluded, now provided as first-class types.
3//!
4//! Regularization changes the fit in ways the OLS formulas can't be reused for.
5//! Under ridge the hat matrix becomes `H_λ = X(XᵀX + λI)⁻¹Xᵀ`, so leverage,
6//! degrees of freedom, and everything built on them differ; lasso has no
7//! closed-form hat matrix at all. Each estimator therefore gets its own fitted
8//! type with the diagnostics that are actually well-defined for it:
9//!
10//! * [`RidgeFit`] — closed-form ridge (via SVD), with **effective degrees of
11//!   freedom** `Σ dⱼ²/(dⱼ²+λ)`, ridge leverage, GCV, effective AIC/BIC, and a
12//!   [`ridge_vif`](RidgeFit::ridge_vif) that generalizes the OLS VIF and reduces
13//!   to it at `λ = 0` — the "VIF before/after regularization" comparison.
14//! * [`LassoFit`] — coordinate-descent lasso, whose natural degrees-of-freedom
15//!   estimate is simply the size of the **active set** (Zou–Hastie–Tibshirani).
16//! * [`ElasticNetFit`] — the lasso/ridge blend
17//!   `λ[α‖β‖₁ + ½(1−α)‖β‖²]`, with a **shrinkage-aware** effective df (the
18//!   active-set trace under the ridge part) that interpolates between the two.
19//! * [`PenalizedLogisticFit`] — a **penalized GLM**: ridge-penalized logistic
20//!   regression by penalized IRLS, with effective df `tr[(XᵀWX+λP)⁻¹XᵀWX]`,
21//!   sandwich standard errors, and shrinkage-aware AIC/BIC.
22//!
23//! ## Penalty conventions (read before comparing `λ` across estimators)
24//!
25//! * **Ridge** penalizes the centered predictors on their given scale; the
26//!   intercept (a detected constant column) is never penalized. Ridge is *not*
27//!   scale-invariant, so standardizing predictors first is the usual practice.
28//! * **Lasso** and **elastic net** standardize predictors internally and
29//!   minimize the `(1/2n)`-scaled objective, so their `λ` is on a different
30//!   scale than ridge's. Elastic net's `α = 1` reproduces the lasso exactly.
31//! * **Penalized logistic** penalizes the coefficients on their given scale
32//!   (like ridge OLS), leaving a detected intercept unpenalized.
33//!
34//! Neither is a drop-in for the other's `λ`; they are documented per-type.
35
36mod elastic_net;
37mod lasso;
38mod penalized_glm;
39mod ridge;
40
41pub use elastic_net::ElasticNetFit;
42pub use lasso::LassoFit;
43pub use penalized_glm::PenalizedLogisticFit;
44pub use ridge::{select_lambda_gcv, RidgeFit};
regression_diagnostics/regularized/mod.rs

regression_diagnostics/regularized/
mod.rs
