Struct DataParallelCoordinator 

pub struct DataParallelCoordinator { /* private fields */ }

Data parallelism coordinator.

Implementations

impl DataParallelCoordinator

pub fn new( config: DistributedConfig, backend: Arc<RwLock<dyn CommunicationBackend>>, ) -> Self

Create a new data parallel coordinator.

pub fn distribute_batch(&self, batch_size: usize) -> Vec<(usize, usize)>

Distribute batch across devices.

pub fn synchronize_gradients(&self) -> Result<(), ExecutorError>

Synchronize gradients across devices.

pub fn devices(&self) -> &[Device]

Get the list of devices.