Struct GradientSparsifier 

pub struct GradientSparsifier {
    pub config: SparsityConfig,
    pub residual: Vec<f32>,
    pub stats: SparsifierStats,
}

Stateful gradient sparsifier with optional residual accumulation.

Residual accumulation ensures that gradient information dropped in one round is carried forward and injected in subsequent rounds, preventing systematic bias toward always-large parameters.

Fields

config: SparsityConfig

Sparsification policy.

residual: Vec<f32>

Accumulated residual from previously dropped gradient elements.

stats: SparsifierStats

Cumulative statistics.

Implementations

impl GradientSparsifier

pub fn new(config: SparsityConfig, gradient_len: usize) -> Self

Create a new sparsifier.

gradient_len is used to pre-allocate the residual buffer. If the gradient length changes between calls, the residual is silently zero-extended or truncated to match.

pub fn sparsify(&mut self, gradient: &[f32]) -> SparseGradient

Sparsify a gradient vector.

Steps performed:

1. If accumulate_residuals, add the stored residual to gradient element-wise (extending or truncating the residual as needed).
2. Apply top-k and/or threshold selection.
3. Update the residual with the dropped elements.
4. Update statistics.

Returns a SparseGradient containing only the kept elements.

pub fn reset_residual(&mut self)

Clear the residual buffer (set all entries to zero).