Expand description
GradientCheckpointer — gradient accumulation, checkpointing, and replay for distributed training with fault tolerance.
This module implements a production-grade gradient checkpointing system that supports configurable accumulation modes (Sum, Mean, WeightedMean), L2-norm gradient clipping, FNV-1a content checksums, and bounded checkpoint history with automatic eviction.
Structs§
- Checkpoint
Id - Newtype wrapper for checkpoint identifiers.
- Checkpointer
Config - Configuration for a
GradientCheckpointer. - GcCheckpointer
Stats - Snapshot of checkpointer statistics.
- GcGradient
Checkpoint - A snapshot of accumulated and (optionally clipped) gradients at a particular training step.
- GcGradient
Tensor - A gradient tensor associated with a named layer at a particular training step.
- Gradient
Checkpointer - Production-grade gradient checkpointing engine.
Enums§
- GcAccumulation
Mode - Controls how multiple gradient tensors for the same layer are combined
during a
flush. - Gradient
Checkpointer Error - Errors produced by
GradientCheckpointer.
Functions§
- fnv1a_
f64_ slice - Compute FNV-1a hash over a slice of
f64values.