Struct GradientUtils 

pub struct GradientUtils;

Utility functions for gradient operations

Implementations

impl GradientUtils

pub fn clip_grad_norm(gradients: &mut Vec<Tensor>, max_norm: f32) -> Result<f32>

Clip gradients by global norm

This function clips the gradients of a list of tensors by their global norm. If the global norm exceeds max_norm, all gradients are scaled down proportionally.

Arguments

• gradients - Mutable reference to a vector of gradients
• max_norm - Maximum allowed norm for gradient clipping

Returns

The actual norm of the gradients before clipping

pub fn clip_grad_value( gradients: &mut Vec<Tensor>, clip_value: f32, ) -> Result<()>

Clip gradients by value

This function clips each gradient tensor element-wise to be within [-clip_value, clip_value]

Arguments

• gradients - Mutable reference to a vector of gradients
• clip_value - Maximum absolute value for each gradient element

pub fn accumulate_gradients( accumulated_grads: &mut Vec<Tensor>, new_grads: &[Tensor], accumulation_steps: usize, ) -> Result<()>

Accumulate gradients by averaging

This function adds new gradients to accumulated gradients and scales them by 1/accumulation_steps to maintain proper scaling

Arguments

• accumulated_grads - Mutable reference to accumulated gradients
• new_grads - New gradients to add
• accumulation_steps - Number of accumulation steps (for scaling)

pub fn zero_accumulated_gradients( accumulated_grads: &mut Vec<Tensor>, ) -> Result<()>

Zero out accumulated gradients