Struct GemmaRmsNorm 

pub struct GemmaRmsNorm { /* private fields */ }

Gemma-style RmsNorm that adds +1.0 to the weight during initialization.

Unlike using RmsNorm::new_gemma(), this type stores the original checkpoint weight separately, ensuring that UQFF serialization (via ToTensors) always returns the un-offset weight. This prevents the double-addition bug where new_gemma would add +1.0 on both write and read.

Implementations

impl GemmaRmsNorm

pub fn new(size: usize, eps: f64, vb: ShardedVarBuilder) -> Result<Self>

pub fn weight(&self) -> &Tensor

pub fn original_weight(&self) -> &Tensor

pub fn eps(&self) -> f64

pub fn forward_residual(&self, x: &Tensor, residual: &Tensor) -> Result<Tensor>

pub fn forward_residual_scaled( &self, x: &Tensor, residual: &Tensor, scale: &Tensor, ) -> Result<Tensor>

Trait Implementations

impl Clone for GemmaRmsNorm

fn clone(&self) -> GemmaRmsNorm

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GemmaRmsNorm

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Module for GemmaRmsNorm

fn forward(&self, x: &Tensor) -> Result<Tensor>