Struct GradientAccumulationConfig

pub struct GradientAccumulationConfig {
    pub accumulation_steps: usize,
    pub average_gradients: bool,
    pub zero_gradients_after_update: bool,
    pub clip_gradients: bool,
    pub max_gradient_norm: Option<f64>,
    pub log_gradient_stats: bool,
}

Gradient accumulation configuration

Fields

accumulation_steps: usize

Number of batches to accumulate gradients over

average_gradients: bool

Whether to average gradients over accumulation steps If true, the gradients are divided by accumulation_steps If false, the gradients are summed over accumulation steps

zero_gradients_after_update: bool

Whether to zero gradients after each update

clip_gradients: bool

Whether to clip gradients

max_gradient_norm: Option<f64>

Maximum gradient norm for gradient clipping

log_gradient_stats: bool

Whether to log gradient statistics during training

Trait Implementations

impl Clone for GradientAccumulationConfig

fn clone(&self) -> GradientAccumulationConfig

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GradientAccumulationConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for GradientAccumulationConfig

fn default() -> Self

Returns the “default value” for a type.