Struct RMSNorm 

pub struct RMSNorm {
    pub config: LayerNormConfig,
}

RMS (Root Mean Square) normalization

A simplified variant of layer normalization that only computes RMS:

RMSNorm(x) = x / RMS(x) * γ
where RMS(x) = √(mean(x²) + ε)

Fields

config: LayerNormConfig

Configuration

Implementations

impl RMSNorm

pub fn new(config: LayerNormConfig) -> Result<Self>

Create a new RMS normalization component

pub fn build_rmsnorm_graph(&self, graph: &mut EinsumGraph) -> Result<Vec<usize>>

Build einsum graph for RMS normalization

Input tensors:

• 0: x (input) [batch, seq_len, d_model]
• 1: gamma (scale) [d_model] (if elementwise_affine)

Output tensors:

• output: [batch, seq_len, d_model] (normalized)

pub fn eps(&self) -> f64

Get epsilon value

Trait Implementations

impl Clone for RMSNorm

fn clone(&self) -> RMSNorm

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RMSNorm

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.