Trait DistributedOptimizer 

pub trait DistributedOptimizer: Optimizer {
    type Communicator;

    // Required methods
    fn all_reduce_gradients(&mut self, gradients: &mut [Tensor]) -> Result<()>;
    fn broadcast_parameters(&mut self, parameters: &mut [Tensor]) -> Result<()>;
    fn rank(&self) -> usize;
    fn world_size(&self) -> usize;
    fn sync_state(&mut self) -> Result<()>;
}

Trait for distributed optimization capabilities.

Provides interfaces for gradient synchronization and distributed training.

Required Associated Types

type Communicator

The communicator type used for distributed operations.

Required Methods

fn all_reduce_gradients(&mut self, gradients: &mut [Tensor]) -> Result<()>

Performs all-reduce operation on gradients.

fn broadcast_parameters(&mut self, parameters: &mut [Tensor]) -> Result<()>

Broadcasts parameters from rank 0 to all other ranks.

fn rank(&self) -> usize

Gets the current rank in the distributed group.

fn world_size(&self) -> usize

Gets the total number of ranks in the distributed group.

fn sync_state(&mut self) -> Result<()>

Synchronizes optimizer state across all ranks.

Implementors