Module gradient_processing

Gradient processing utilities for machine learning optimization

This module provides comprehensive gradient manipulation utilities including various clipping strategies, normalization, and other processing techniques.

Structs

GradientAccumulator

Gradient accumulation utility

GradientClipConfig

Gradient clipping configuration

GradientMask

Gradient masking and freezing utilities

GradientProcessor

Gradient clipping processor

Functions

adaptive_gradient_clipping

Adaptive gradient clipping

add_gradient_noise

Add noise to gradients for regularization

clip_gradient_l1_norm

Clip gradient L1 norm

clip_gradient_norm

Clip gradient L2 norm (global gradient clipping)

clip_gradients_by_value

Clip gradient values to a specified range

gradient_centralization

Compute gradient centralization

zero_small_gradients

Zero out small gradient values