Struct Regularizer

pub struct Regularizer<const P: usize> { /* private fields */ }

A weighted regularization term.

This can be interpreted as a prior of the form

f(\vec{x}) = \frac{p\lambda^{1/p}}{2\Gamma(1/p)}e^{-\frac{\lambda|\vec{x}|^p}}

which becomes a Laplace distribution for $p=1$ and a Gaussian for $p=2$. These are commonly interpreted as $\ell_p$ regularizers for linear regression models, with $p=1$ and $p=2$ corresponding to LASSO and ridge regression, respectively. When used in nonlinear regression, these should be interpeted as the prior listed above when used in maximum a posteriori (MAP) estimation. Explicitly, when the logarithm is taken, this term becomes

\lambda \left(\sum_{j} w_j |x_j|^p\right)^{1/p}

plus some additional constant terms which do not depend on free parameters.

Weights can be specified to vary the influence of each parameter used in the regularization. These weights are typically assigned by first fitting without a regularization term to obtain parameter values $\vec{\beta}$, choosing a value $\gamma>0$, and setting the weights to $\vec{w} = 1/|\vec{\beta}|^\gamma$ according to a paper by Zou¹.

---

1. Zou, H. (2006). The Adaptive Lasso and Its Oracle Properties. In Journal of the American Statistical Association (Vol. 101, Issue 476, pp. 1418–1429). Informa UK Limited. ↩

Implementations

impl<const P: usize> Regularizer<P>

pub fn new<T, U, F>( parameters: T, lambda: f64, weights: Option<F>, ) -> Result<Box<Regularizer<P>>, LadduError>

where T: IntoIterator<Item = U>, U: AsRef<str>, F: AsRef<[f64]>,

Create a new Regularizer term from a list of parameter names, a nonnegative regularization parameter $\lambda$, and an optional list of weights which scale the influence of each parameter in the regularization term. If not set, the weights will default to unity.

Errors

This method will return a LadduError if the number of parameters and weights are not equal.

Trait Implementations

impl<const P: usize> Clone for Regularizer<P>

fn clone(&self) -> Regularizer<P>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl LikelihoodTerm for Regularizer<1>

fn evaluate(&self, parameters: &[f64]) -> f64

Evaluate the term given some input parameters.

fn evaluate_gradient( &self, parameters: &[f64], ) -> Matrix<f64, Dyn, Const<1>, VecStorage<f64, Dyn, Const<1>>>

Evaluate the gradient of the term given some input parameters.

fn parameters(&self) -> Vec<String>

The list of names of the input parameters for LikelihoodTerm::evaluate.

impl LikelihoodTerm for Regularizer<2>

fn evaluate(&self, parameters: &[f64]) -> f64

Evaluate the term given some input parameters.

fn evaluate_gradient( &self, parameters: &[f64], ) -> Matrix<f64, Dyn, Const<1>, VecStorage<f64, Dyn, Const<1>>>

Evaluate the gradient of the term given some input parameters.

fn parameters(&self) -> Vec<String>

The list of names of the input parameters for LikelihoodTerm::evaluate.