Struct AllReduceAggregator 

pub struct AllReduceAggregator { /* private fields */ }

All-reduce aggregator: averages gradients across all workers via ring reduce.

Each worker independently computes local gradients and then calls GradientAggregator::aggregate. Internally the aggregator serialises the gradient tensors, performs a ring all-reduce (scatter-reduce followed by all-gather) over the configured Transport, and returns the element-wise average. This is the most common strategy for synchronous data-parallel training because every worker ends up with the same averaged gradients without a central bottleneck.

Example

let transports = InProcessTransport::create_group(2);
let cfg = DistributedConfig { world_size: 2, rank: 0, coordinator_addr: String::new() };
let mut agg = AllReduceAggregator::new(cfg, Box::new(transports.into_iter().next().unwrap()));
let averaged = agg.aggregate(&local_grads).unwrap();

Implementations

impl AllReduceAggregator

pub fn new(config: DistributedConfig, transport: Box<dyn Transport>) -> Self

Trait Implementations

impl GradientAggregator for AllReduceAggregator

fn aggregate( &mut self, local_gradients: &[Tensor], ) -> Result<Vec<Tensor>, ModelError>

Aggregate local gradients, returning the combined result.