Enum NormFunction 

pub enum NormFunction {
    Rms,
    LayerNorm,
    LayerNormBias,
    LayerNormNoParams,
    RmsNoParams,
    RmsBias,
}

The norm FUNCTION an architecture applies at its parametric sites – the LLM_NORM / LLM_NORM_RMS argument of build_norm, plus whether there is a weight to hand it.

One answer per architecture, read at every site. The loader used to decide this with a chain of if branches restated at the pre-attention, pre-FFN and final sites: three places that had to agree about one thing. crate::norm_sites resolves it ONCE through norm_function and the three sites read the result.

Variants

Rms

LLM_NORM_RMS with a weight. The generic path’s default.

LayerNorm

LLM_NORM with a weight and no bias: dbrx (capability::WEIGHTED_LAYER_NORM).

LayerNormBias

LLM_NORM with a weight AND a bias: orion, nemotron (capability::BIASED_LAYER_NORM), NormOp::LayerNormBias.

LayerNormNoParams

LLM_NORM with neither: olmo (capability::NON_PARAMETRIC_LAYER_NORM).

RmsNoParams

LLM_NORM_RMS with a null weight: talkie (capability::NON_PARAMETRIC_RMS_NORM), NormOp::RmsNoParams.

RmsBias

LLM_NORM_RMS with a weight AND a bias: phimoe (capability::BIASED_RMS_NORM), NormOp::RmsBias.

Implementations

impl NormFunction

pub fn resolve<E>( self, load: impl FnMut(NormParam) -> Result<Vec<f32>, E>, ) -> Result<NormOp, E>

The NormOp for one site, reading its weight through load only when this function has a weight to read.

The closure rather than a Vec<f32> argument is the point: LayerNormNoParams never calls it, so a file that carries no norm tensor (OLMo-1 ships none) is never asked for one, and a site cannot be handed a weight its function would drop.

load is asked for each PART the function has – Weight, and for the biased form Bias too – so a function that has no bias never asks the file for one and the biased form cannot be built with the bias forgotten.

Trait Implementations

impl Clone for NormFunction

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for NormFunction

impl Debug for NormFunction

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for NormFunction

impl PartialEq for NormFunction

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for NormFunction