Module gradient_sparsify 

Gradient sparsification and delta encoding for federated learning.

This module provides bandwidth-efficient gradient transmission primitives:

• SparsityConfig — policy for top-k selection and threshold filtering
• SparseGradient — compact index/value representation with residual support
• GradientSparsifier — stateful sparsifier with residual accumulation
• GradientDelta — delta-encoded gradient relative to the previous round
• DeltaEncoder — stateful encoder that tracks the previously sent gradient

Design rationale

In bandwidth-constrained federated learning scenarios, transmitting the full gradient vector each round wastes network capacity. Two complementary techniques address this:

1. Sparsification — keep only the top-k elements (by absolute value) or those exceeding a magnitude threshold, accumulating the dropped portion in a residual buffer so that no information is permanently lost.

2. Delta encoding — transmit the element-wise difference from the previous round instead of the full gradient; the receiver reconstructs the current gradient by adding the delta to its locally cached copy.

Structs

DeltaEncoder

Stateful encoder that computes element-wise deltas between successive gradient rounds.

DeltaStats

Cumulative statistics for a DeltaEncoder.

GradientDelta

A gradient update that is either a complete gradient or a delta from the previous round.

GradientSparsifier

Stateful gradient sparsifier with optional residual accumulation.

SparseGradient

A compact sparse representation of a gradient vector.

SparsifierStats

Cumulative statistics for a GradientSparsifier.

SparsityConfig

Configuration for the GradientSparsifier.