Struct LayerNormalization 

pub struct LayerNormalization<T: Float + Debug + ScalarOperand + Send + Sync + 'static> { /* private fields */ }

Layer normalization implementation

Implementations

impl<T: Float + Debug + ScalarOperand + Send + Sync + 'static> LayerNormalization<T>

pub fn new(dimension: usize) -> Result<Self>

Create new layer normalization

pub fn forward(&self, input: &Array2<T>) -> Result<Array2<T>>

Forward pass through layer normalization.

Rejects inputs whose width differs from the configured dimension instead of indexing out of bounds.

pub fn dimension(&self) -> usize

Normalized dimension

pub fn backward( &mut self, input: &Array2<T>, grad_output: &Array2<T>, learning_rate: T, ) -> Result<Array2<T>>

Backward pass through layer normalization.

With xhat = (x - mu) / sqrt(var + eps) and y = gamma * xhat + beta, the exact input gradient for one row is dL/dx = (1/sigma) * (g' - mean(g') - xhat * mean(g' * xhat)) where g' = dL/dy * gamma. gamma and beta are updated in place with a plain SGD step and the input gradient is returned.

pub fn parameter_count(&self) -> usize

Get parameter count

pub fn reset(&mut self) -> Result<()>

Reset normalization parameters