Struct ActivityRegularization 

pub struct ActivityRegularization<A: Float + FromPrimitive + Debug> {
    pub lambda: A,
    pub norm: ActivityNorm,
}

Activity regularization

Activity regularization penalizes high activation values in neural networks. It encourages sparse activations by adding a penalty based on the magnitude of activation values.

Parameters

• lambda: Regularization strength parameter
• norm: Type of norm to use for measuring activation magnitudes (L1, L2, or L2Squared)

References

• Nowlan, S. J., & Hinton, G. E. (1992). Simplifying neural networks by soft weight-sharing. Neural Computation, 4(4), 473-493.

Fields

lambda: A

Regularization strength

norm: ActivityNorm

Norm to use for activity regularization

Implementations

impl<A: Float + FromPrimitive + Debug + Send + Sync> ActivityRegularization<A>

pub fn l1(lambda: A) -> Self

Create a new activity regularizer with L1 norm

Arguments

• lambda - Regularization strength parameter

Returns

A new activity regularizer with L1 norm

pub fn l2(lambda: A) -> Self

Create a new activity regularizer with L2 norm

Arguments

• lambda - Regularization strength parameter

Returns

A new activity regularizer with L2 norm

pub fn l2_squared(lambda: A) -> Self

Create a new activity regularizer with squared L2 norm

Arguments

• lambda - Regularization strength parameter

Returns

A new activity regularizer with squared L2 norm

pub fn new(lambda: A, norm: ActivityNorm) -> Self

Create a new activity regularizer with custom norm

Arguments

• lambda - Regularization strength parameter
• norm - Norm to use (L1, L2, or L2Squared)

Returns

A new activity regularizer with specified norm

Trait Implementations

impl<A: Clone + Float + FromPrimitive + Debug> Clone for ActivityRegularization<A>

fn clone(&self) -> ActivityRegularization<A>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<A: Debug + Float + FromPrimitive + Debug> Debug for ActivityRegularization<A>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<A, D> Regularizer<A, D> for ActivityRegularization<A>

where A: Float + ScalarOperand + Debug + FromPrimitive + Send + Sync, D: Dimension,

fn apply(&self, params: &Array<A, D>, gradients: &mut Array<A, D>) -> Result<A>

Apply regularization to parameters and gradients

fn penalty(&self, params: &Array<A, D>) -> Result<A>

Compute the regularization penalty value

impl<A: Copy + Float + FromPrimitive + Debug> Copy for ActivityRegularization<A>