Struct RmsNorm 

pub struct RmsNorm {
    pub normalized_shape: Vec<usize>,
    pub eps: f64,
    pub gamma: ArrayD<f64>,
}

Root Mean Square Layer Normalization (no mean centering).

Normalizes by: x / sqrt(mean(x^2) + eps) * gamma

Used in LLaMA and other modern transformer architectures.

Fields

normalized_shape: Vec<usize>

Shape of the dimensions to normalize over (typically the last N dims).

eps: f64

Small constant for numerical stability.

gamma: ArrayD<f64>

Learnable scale parameter.

Implementations

impl RmsNorm

pub fn new( normalized_shape: Vec<usize>, eps: f64, ) -> Result<Self, NormalizationError>

Create a new RmsNorm layer.

normalized_shape specifies the trailing dimensions to normalize over. gamma is initialized to ones.

pub fn forward( &self, input: &ArrayD<f64>, ) -> Result<ArrayD<f64>, NormalizationError>

Forward pass: normalize the input tensor.

The last normalized_shape.len() dimensions of the input must match normalized_shape.

pub fn rms(input: &ArrayD<f64>, axes: &[usize]) -> ArrayD<f64>

Compute Root Mean Square along specified axes (keeping dims).

pub fn update_gamma( &mut self, new_gamma: ArrayD<f64>, ) -> Result<(), NormalizationError>

Update the learnable scale parameter gamma.

Trait Implementations

impl Clone for RmsNorm

fn clone(&self) -> RmsNorm

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RmsNorm

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.