Struct LayerNormLayer 

pub struct LayerNormLayer {
    pub n_features: usize,
    pub gamma: Vec<f64>,
    pub beta: Vec<f64>,
    pub epsilon: f64,
}

Layer normalisation (Ba et al., 2016).

Normalises the entire feature vector of a single sample to zero mean and unit variance, then applies learned scale (gamma) and shift (beta).

Fields

n_features: usize

Number of features (last dimension size).

gamma: Vec<f64>

Learned scale parameter (gamma), initialised to 1.

beta: Vec<f64>

Learned shift parameter (beta), initialised to 0.

epsilon: f64

Numerical stability constant.

Implementations

impl LayerNormLayer

pub fn new(n_features: usize) -> Self

Create a new LayerNorm with identity transform (gamma=1, beta=0).

pub fn forward(&self, input: &[f64]) -> Vec<f64>

Apply layer normalisation to one sample vector.

output[i] = gamma[i] * (input[i] - mean) / sqrt(var + eps) + beta[i]

pub fn backward( &self, input: &[f64], d_output: &[f64], ) -> (Vec<f64>, Vec<f64>, Vec<f64>)

Compute gradient of the layer norm output with respect to the input.

Returns (d_input, d_gamma, d_beta) given upstream gradient d_output.

Trait Implementations

impl Clone for LayerNormLayer

fn clone(&self) -> LayerNormLayer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LayerNormLayer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.