Trait Communicator 

pub trait Communicator: Send {
    // Required methods
    fn info(&self) -> CommInfo;
    fn all_reduce(&self, tensor: &Tensor, op: ReduceOp) -> Result<Tensor>;
    fn all_gather(&self, tensor: &Tensor, dim: usize) -> Result<Tensor>;
    fn reduce_scatter(
        &self,
        tensor: &Tensor,
        op: ReduceOp,
        dim: usize,
    ) -> Result<Tensor>;
    fn broadcast(&self, tensor: &Tensor, root: usize) -> Result<Tensor>;
    fn barrier(&self) -> Result<()>;
}

Abstraction over collective communication backends.

Implementations wrap NCCL, Gloo, MPI, or mock backends. All operations are synchronous — they block until the collective is complete.

The trait is object-safe so it can be stored as Box<dyn Communicator>.

Every shard handed to an implementation must belong to the same graph as the value it is combined with. Tensor ops are graph-affine: two tensors can be combined iff their graph_ids match. An out-of-process implementation that receives bytes over the wire must therefore materialize them onto the caller’s graph (const_f32_like, from_f32_on), not mint a fresh one.

Required Methods

fn info(&self) -> CommInfo

Communicator info (rank, world size).

fn all_reduce(&self, tensor: &Tensor, op: ReduceOp) -> Result<Tensor>

All-reduce: compute element-wise reduction across all ranks, distributing the result to every rank.

fn all_gather(&self, tensor: &Tensor, dim: usize) -> Result<Tensor>

All-gather: concatenate tensors from all ranks along dim.

fn reduce_scatter( &self, tensor: &Tensor, op: ReduceOp, dim: usize, ) -> Result<Tensor>

Reduce-scatter: reduce across ranks then scatter equal chunks to each rank.

fn broadcast(&self, tensor: &Tensor, root: usize) -> Result<Tensor>

Broadcast tensor from root to all ranks.

fn barrier(&self) -> Result<()>

Barrier: block until all ranks reach this point.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl Communicator for IdentityComm