Struct ShakeDrop 

pub struct ShakeDrop<A: Float + FromPrimitive + Debug> {
    pub p: A,
    pub alpha_range: (A, A),
    pub beta_range: (A, A),
    /* private fields */
}

ShakeDrop regularization

ShakeDrop is a regularization method that extends Stochastic Depth and is often used in very deep neural networks. It randomly scales activations during training.

Parameters

• p - The probability of activating ShakeDrop (probability of activating the forward pass transformation), value between 0 and 1.
• alpha_range - The range for the alpha parameter used in forward pass (default: [-1.0, 1.0]).
• beta_range - The range for the beta parameter used in backward pass (default: [0.0, 1.0]).

References

• Yamada, Y., Iwamura, M., & Kise, K. (2018). ShakeDrop regularization. arXiv preprint arXiv:1802.02375.

Fields

p: A

Probability of applying ShakeDrop

alpha_range: (A, A)

Range for the alpha parameter

beta_range: (A, A)

Range for the beta parameter

Implementations

impl<A: Float + FromPrimitive + Debug + Send + Sync> ShakeDrop<A>

pub fn new(p: A) -> Self

Create a new ShakeDrop regularizer

Arguments

• p - Probability of applying ShakeDrop, between 0 and 1
• alpha_range - Range for the alpha parameter (default: [-1.0, 1.0])
• beta_range - Range for the beta parameter (default: [0.0, 1.0])

Returns

A ShakeDrop regularizer

pub fn new_with_ranges(p: A, alpharange: (A, A), beta_range: (A, A)) -> Self

Create a new ShakeDrop regularizer with custom ranges

Arguments

• p - Probability of applying ShakeDrop, between 0 and 1
• alpha_range - Range for the alpha parameter
• beta_range - Range for the beta parameter

Returns

A ShakeDrop regularizer

pub fn forward<S, D>(&mut self, x: &ArrayBase<S, D>) -> (Array<A, D>, (A, A, A))

where S: Data<Elem = A>, D: Dimension,

Apply ShakeDrop to input activations

Arguments

• x - Input activation tensor

Returns

The transformed activations and gate parameters for use in backward pass

pub fn backward<S, D>( &self, grad_output: &ArrayBase<S, D>, gate_params: (A, A, A), ) -> Array<A, D>

where S: Data<Elem = A>, D: Dimension,

Backward pass for ShakeDrop

Arguments

• grad_output - Gradient from the next layer
• gate_params - The gate parameters (b, alpha, beta) from the forward pass

Returns

The modified gradients

Trait Implementations

impl<A: Clone + Float + FromPrimitive + Debug> Clone for ShakeDrop<A>

fn clone(&self) -> ShakeDrop<A>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<A: Debug + Float + FromPrimitive + Debug> Debug for ShakeDrop<A>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<A: Float + FromPrimitive + Debug + ScalarOperand, D: Dimension + Send + Sync> Regularizer<A, D> for ShakeDrop<A>

fn apply(&self, _params: &Array<A, D>, gradients: &mut Array<A, D>) -> Result<A>

Apply regularization to parameters and gradients

fn penalty(&self, params: &Array<A, D>) -> Result<A>

Compute the regularization penalty value