Struct DataParallelModel 

pub struct DataParallelModel { /* private fields */ }

Data-parallel wrapper around a named weight map.

Simulates splitting a mini-batch across world_size workers, each computing local gradients, then performing an AllReduce followed by an SGD update. Because all workers live in the same process they share a single Arc<RwLock<HashMap>> so weight broadcasts are free.

Implementations

impl DataParallelModel

pub fn new( weights: HashMap<String, Vec<f32>>, config: DistributedConfig, ) -> Self

Create a new data-parallel model with the given weight map and config.

pub fn weights(&self) -> HashMap<String, Vec<f32>>

Return a snapshot of the current weight map.

pub fn step( &self, local_grads: Vec<GradientBuffer>, learning_rate: f32, ) -> ModelResult<()>

Apply a gradient update using the configured strategy.

For AllReduce with world_size > 1 this pushes local gradients to the SharedGradientStore and then applies the averaged result. For single- worker or NoSync modes the update is applied directly.

Errors

Propagates gradient-store and weight-lock errors.

pub fn broadcast_weights(&self) -> ModelResult<()>

Broadcast weights from rank 0 to all ranks.

In-process: all workers already share the same Arc, so this is a no-op that succeeds immediately.