Skip to main content

Module ard

blr_core

Module ard 

Source
Expand description

BLR+ARD EM fitting, prediction API, and configuration.

Implements MacKay (1992) / Tipping (2001) empirical Bayes for Bayesian Linear Regression with Automatic Relevance Determination.

§Overview

The main entry point is fit, which runs an EM loop to find the posterior distribution over weights and the noise precision hyperparameter β. ARD places an independent precision hyperparameter α_d on each weight; features with low signal converge to α_d → ∞, effectively removing them from the model.

After fitting, use FittedArd to:

§Example: Basic Fit

use blr_core::{fit, ArdConfig};

// 20 observations, 3 features (row-major feature matrix)
let phi: Vec<f64> = vec![1.0; 60];
let y:   Vec<f64> = vec![0.5; 20];
let config = ArdConfig::default();

let fitted = fit(&phi, &y, 20, 3, &config)
    .expect("fit should succeed with valid input");

assert!(fitted.noise_std() > 0.0);
assert_eq!(fitted.relevant_features(None).len(), 3);

§Example: Inspect ARD Relevance

use blr_core::{fit, ArdConfig};

let phi: Vec<f64> = vec![1.0; 60];
let y:   Vec<f64> = vec![0.5; 20];
let fitted = fit(&phi, &y, 20, 3, &ArdConfig::default()).unwrap();

// relevance() returns 1/αd — larger means more relevant
let rel = fitted.relevance();
println!("Feature relevances: {:?}", rel);

// relevant_features() returns a boolean mask
let active = fitted.relevant_features(None);
let n_active = active.iter().filter(|&&x| x).count();
println!("{} of {} features are active", n_active, active.len());

§EM Algorithm Summary

Each iteration:

  1. E-step: Compute posterior mean μ and covariance Σ using the current {α_d, β}.
  2. M-step: Update each α_d and optionally β using the posterior statistics (gamma updates from MacKay 1992 Eq. 32–33).
  3. Convergence: Stop when the change in log-evidence between consecutive iterations is below ArdConfig::tol, or max_iter is reached.

§References

  • MacKay, D. J. C. (1992). “Bayesian Interpolation.” Neural Computation, 4(3), 415–447.
  • Tipping, M. E. (2001). “Sparse Bayesian Learning and the Relevance Vector Machine.” Journal of Machine Learning Research, 1, 211–244.

Structs§

ArdConfig
Configuration for the BLR+ARD EM fitting loop.
BLRPrior
Custom prior for BLR+ARD fitting (batch transfer learning).
FittedArd
Result of a successful fit() call.
PredictiveMarginals
Marginal predictive distributions for a set of test points.

Functions§

fit
Fit BLR+ARD via EM (Type-II maximum likelihood / empirical Bayes).
fit_with_prior
Fit BLR+ARD with an optional informed prior (batch transfer learning).