Struct WeightStandardization 

pub struct WeightStandardization<A: Float> { /* private fields */ }

Weight Standardization

Weight Standardization normalizes the weights along the channel dimension by adjusting them to have zero mean and unit variance. This helps with training stability, especially when used with batch normalization.

Example

use scirs2_core::ndarray::array;
use optirs_core::regularizers::{WeightStandardization, Regularizer};

let weight_std = WeightStandardization::new(1e-5);
let weights = array![[1.0, 2.0], [3.0, 4.0]];
let mut gradients = array![[0.1, 0.2], [0.3, 0.4]];

// Get standardized weights
let standardized = weight_std.standardize(&weights).unwrap();

// Apply during training (modifies gradients)
let _ = weight_std.apply(&weights, &mut gradients);

Implementations

impl<A: Float + Debug + ScalarOperand + FromPrimitive + Send + Sync> WeightStandardization<A>

pub fn new(eps: f64) -> Self

Create a new Weight Standardization regularizer

Arguments

• eps - Small constant for numerical stability (typically 1e-5)

pub fn standardize(&self, weights: &Array2<A>) -> Result<Array2<A>>

Apply weight standardization to a 2D weight matrix

Standardizes the weights to have zero mean and unit variance.

Arguments

• weights - 2D weight matrix

Returns

Standardized weights with zero mean and unit variance

pub fn standardize_conv4d(&self, weights: &Array4<A>) -> Result<Array4<A>>

Apply weight standardization to 4D convolutional weights

Arguments

• weights - Convolutional weights with shape [out_channels, in_channels, height, width]

Returns

Standardized convolutional weights

Trait Implementations

impl<A: Clone + Float> Clone for WeightStandardization<A>

fn clone(&self) -> WeightStandardization<A>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<A: Debug + Float> Debug for WeightStandardization<A>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<A: Float + Debug + ScalarOperand + FromPrimitive + Send + Sync, D: Dimension + Send + Sync> Regularizer<A, D> for WeightStandardization<A>

fn apply(&self, params: &Array<A, D>, gradients: &mut Array<A, D>) -> Result<A>

Apply regularization to parameters and gradients

fn penalty(&self, params: &Array<A, D>) -> Result<A>

Compute the regularization penalty value