Struct ProcessGroup 

pub struct ProcessGroup { /* private fields */ }

A handle to the collective-communication world: a rank, a size, and the transport that connects them.

Wraps an Arc<dyn Transport> so the same group can be cloned into the generator loop, the weight loader, and (later) per-layer collective ops without re-establishing connections.

Implementations

impl ProcessGroup

pub fn new(transport: Arc<dyn Transport>) -> ProcessGroup

pub fn rank(&self) -> u32

pub fn world_size(&self) -> u32

pub fn is_leader(&self) -> bool

true on the lowest rank — the canonical place to print, sample, or own the final logits in a pipeline.

pub fn transport(&self) -> &Arc<dyn Transport>

pub fn barrier(&self) -> Result<(), CollectiveError>

pub fn send_f32( &self, to: u32, tag: u32, data: &[f32], ) -> Result<(), CollectiveError>

Send a hidden-state (or any) tensor to to. tag must be below TAG_RESERVED_BASE (those tags are reserved for collectives).

pub fn recv_f32(&self, from: u32, tag: u32) -> Result<Vec<f32>, CollectiveError>

Receive a tensor from from. Length is taken from the wire.

pub fn all_reduce( &self, data: &mut [f32], op: ReduceKind, ) -> Result<(), CollectiveError>

In-place all-reduce: on return every rank holds op({each rank's input}). All ranks must pass the same length.

Gather-to-root: non-roots ship their vector to rank 0, rank 0 reduces and ships the result back. O(n) at the root — fine for a handful of nodes; swap in ring-reduce later without touching callers.

pub fn all_gather(&self, local: &[f32]) -> Result<Vec<f32>, CollectiveError>

All-gather: every rank contributes local (same length on every rank) and receives the concatenation in rank order, length world_size * local.len().

pub fn broadcast( &self, root: u32, data: &mut [f32], ) -> Result<(), CollectiveError>

Broadcast data from root to every rank. On non-root ranks data is overwritten; its length must already match the root’s.

Trait Implementations

impl Clone for ProcessGroup

fn clone(&self) -> ProcessGroup

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.