Enum LossScaler 

pub enum LossScaler {
    None,
    Static {
        scale: f32,
    },
    Dynamic {
        scale: f32,
        growth_factor: f32,
        backoff_factor: f32,
        growth_interval: usize,
        steps_since_overflow: usize,
    },
}

Loss scaling strategy for mixed precision training.

Variants

None

No loss scaling (not recommended for FP16)

Static

Static loss scaling with fixed scale factor

Fields

scale: f32

Dynamic

Dynamic loss scaling that adjusts based on gradient overflow

Fields

scale: f32

Current scale factor

growth_factor: f32

Growth factor when no overflow (typically 2.0)

backoff_factor: f32

Backoff factor when overflow detected (typically 0.5)

growth_interval: usize

Number of successful steps before growing scale

steps_since_overflow: usize

Current step counter

Implementations

impl LossScaler

pub fn static_scale(scale: f32) -> Self

Creates a static loss scaler.

pub fn dynamic( initial_scale: f32, growth_factor: f32, growth_interval: usize, ) -> Self

Creates a dynamic loss scaler with typical defaults.

Arguments

• initial_scale - Starting scale (typically 2^15 = 32768)
• growth_factor - How much to grow scale (typically 2.0)
• growth_interval - Steps before growing (typically 2000)

pub fn get_scale(&self) -> f32

Gets the current scale factor.

pub fn scale_loss(&self, loss: f32) -> f32

Scales a loss value.

pub fn unscale_gradients(&self, gradients: &mut Array2<f32>)

Unscales gradients (divides by scale).

pub fn update(&mut self, overflow_detected: bool) -> bool

Updates the dynamic scaler based on overflow detection.

Arguments

• overflow_detected - Whether gradient overflow was detected

Returns

True if the optimizer step should proceed

Trait Implementations

impl Clone for LossScaler

fn clone(&self) -> LossScaler

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LossScaler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for LossScaler

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for LossScaler

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.