Struct RowParallelLinear 

pub struct RowParallelLinear { /* private fields */ }

Splits the input dimension across n_workers.

Each worker holds weights [n_in/n_workers, n_out]. The partial results are summed (all-reduce) and the shared bias is added once.

Implementations

impl RowParallelLinear

pub fn new( n_in: usize, n_out: usize, config: TensorParallelConfig, seed: u64, ) -> NeuralResult<Self>

Create a row-parallel linear layer.

Errors

• n_in is not divisible by config.n_workers.
• config.n_workers == 0.

pub fn forward(&self, input: &Array2<f64>) -> NeuralResult<Array2<f64>>

Forward pass.

Each worker computes y_i = input_i @ W_i where input_i = input[:, i*chunk:(i+1)*chunk]. The partial products are summed and the bias is added: y = Σ y_i + bias.

pub fn n_in(&self) -> usize

Total input features (across all workers).