Struct WeightNorm 

pub struct WeightNorm {
    pub dim: usize,
}

Weight Normalization: reparametrizes weight as w = g * v / ||v||.

This is a weight reparametrization technique, not a layer normalization.

Fields

dim: usize

Dimension along which to compute the norm.

Implementations

impl WeightNorm

pub fn new(dim: usize) -> Self

Create a new WeightNorm reparametrization.

pub fn apply( &self, weight: &ArrayD<f64>, ) -> Result<(ArrayD<f64>, ArrayD<f64>), NormalizationError>

Decompose a weight tensor into (g, v) where g = ||w|| per slice along self.dim and v = w / ||w||.

pub fn reparametrize( g: &ArrayD<f64>, v: &ArrayD<f64>, dim: usize, ) -> Result<ArrayD<f64>, NormalizationError>

Reparametrize: given (g, v) compute g * v / ||v||.

Trait Implementations

impl Clone for WeightNorm

fn clone(&self) -> WeightNorm

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for WeightNorm

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.