Skip to main content

py_regularizer

Function py_regularizer 

Source
pub fn py_regularizer(
    parameters: Vec<String>,
    lda: f64,
    p: usize,
    weights: Option<Vec<f64>>,
) -> PyResult<PyLikelihoodExpression>
Expand description

An weighted :math:\ell_p regularization term which acts as a maximum a posteriori (MAP) prior.

This can be interpreted as a prior of the form

.. math:: f(\vec{x}) = \frac{p\lambda^{1/p}}{2\Gamma(1/p)}e^{-\lambda|\vec{x}|^p}

which becomes a Laplace distribution for :math:p=1 and a Gaussian for :math:p=2. These are commonly interpreted as :math:\ell_p regularizers for linear regression models, with :math:p=1 and :math:p=2 corresponding to LASSO and ridge regression, respectively. When used in nonlinear regression, these should be interpeted as the prior listed above when used in maximum a posteriori (MAP) estimation. Explicitly, when the logarithm is taken, this term becomes

.. math:: \lambda \left(\sum_{j} w_j |x_j|^p\right)^{1/p}

plus some additional constant terms which do not depend on free parameters.

Weights can be specified to vary the influence of each parameter used in the regularization. These weights are typically assigned by first fitting without a regularization term to obtain parameter values :math:\vec{\beta}, choosing a value :math:\gamma>0, and setting the weights to :math:\vec{w} = 1/|\vec{\beta}|^\gamma according to [Zou]_.

§References

.. [Zou] Zou, H. (2006). The Adaptive Lasso and Its Oracle Properties. Journal of the American Statistical Association, 101(476), 1418–1429. doi:10.1198/016214506000000735

§Parameters

parameters : list of str The names of the parameters to regularize lda : float The regularization parameter :math:\lambda p : {1, 2} The degree of the norm :math:\ell_p weights : list of float, optional Weights to apply in the regularization to each parameter

§Raises

ValueError If :math:p is not 1 or 2 Exception If the number of parameters and weights is not equal

§Returns

LikelihoodExpression A term that can be combined with other likelihood expressions.