Struct RmsNorm 

pub struct RmsNorm {
    pub weight: Array,
    pub eps: f32,
}

Root-mean-square normalisation over the last axis with a learned scale (MLX’s nn.RMSNorm, the fused fast::rms_norm).

Fields

weight: Array

The scale, [dims], initially ones.

eps: f32

Added to the mean square.

Implementations

impl RmsNorm

pub fn new(dims: usize, eps: f32) -> Result<Self>

A layer over dims features (MLX’s default eps is 1e-5).

pub fn forward(&self, x: &Array) -> Result<Array>

The normalised x.

Trait Implementations

impl Clone for RmsNorm

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RmsNorm

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Module for RmsNorm

fn visit(&mut self, p: &mut Parameters<'_>)

Reports every parameter and submodule to p, frozen ones included.

fn parameters(&self) -> Vec<(String, Array)>

Every parameter with its path (layers.0.weight), in visiting order.

fn trainable_parameters(&self) -> Vec<(String, Array)>

The parameters that are trained (not frozen), with their paths.

fn update_trainable(&mut self, values: &[Array]) -> Result<()>

Replaces the trainable parameters, given in the order of Module::trainable_parameters (as optimizers return them).

fn load_weights( &mut self, weights: impl IntoIterator<Item = (String, Array)>, strict: bool, ) -> Result<()>

Replaces parameters by path, as MLX’s load_weights: each weight must have its parameter’s shape (the dtype is kept as loaded). With strict, every parameter must be given and no unknown names are allowed.