Struct RowParallel 

pub struct RowParallel<C: Communicator> { /* private fields */ }

Row-parallel linear layer.

Each rank holds rows [rank * in/N .. (rank+1) * in/N] of the weight. Forward pass: Y_local = X_local @ W_local, then AllReduce(Sum) to combine.

This is the SPMD form

It reduces via a Communicator, so it describes one rank’s half of the work. Fuel ships no multi-process Communicator (see crate::comm), so with the only in-tree impl — IdentityComm, world_size == 1 — the all-reduce is an identity and the result is the single rank’s own product. That is correct for one rank and is not a multi-device reduction.

For one process driving N devices, hold all N partials and fold them with DeviceGroup::all_reduce, which is a real cross-device collective.

Implementations

impl<C: Communicator> RowParallel<C>

pub fn new(linear: Linear, shard: TensorShard, comm: C) -> Self

Wrap a pre-sharded linear layer with a communicator for all-reduce.

pub fn shard(&self) -> &TensorShard

Shard metadata.

pub fn forward(&self, x: &Tensor) -> Result<Tensor>

Forward: local matmul then all-reduce sum.