Skip to main content

DistributedBackend

pmetal_distributed

Trait DistributedBackend 

Source 
pub trait DistributedBackend: Send + Sync {
    // Required methods
    fn rank(&self) -> usize;
    fn world_size(&self) -> usize;
    fn all_reduce<'life0, 'life1, 'async_trait>(
        &'life0 self,
        buffer: &'life1 mut [u8],
    ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
    fn barrier<'life0, 'async_trait>(
        &'life0 self,
    ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
}
Expand description

Interface for distributed operations.

Required Methods§

Source

fn rank(&self) -> usize

Get the rank of this node (0 to world_size - 1).

Source

fn world_size(&self) -> usize

Get the total number of nodes.

Source

fn all_reduce<'life0, 'life1, 'async_trait>( &'life0 self, buffer: &'life1 mut [u8], ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Perform an all-reduce operation on a buffer (sum).

The input buffer contains the local gradients. On return, it should contain the sum of gradients from all nodes.

Source

fn barrier<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Barrier synchronization.

Implementors§