Struct ColumnParallelLinear 

pub struct ColumnParallelLinear {
    pub weight: Parameter,
    pub bias: Option<Parameter>,
    pub rank: usize,
    pub world_size: usize,
    pub barrier: Arc<CollectiveBarrier>,
}

Linear layer with weight sharded along columns (N dimension).

Forward: Y_t = X @ W_t (no collective). Backward: grad_X = AllReduce(grad_Y_t @ W_t^T) — handled by normal autograd + external AllReduce call.

Fields

weight: Parameter

bias: Option<Parameter>

rank: usize

world_size: usize

barrier: Arc<CollectiveBarrier>

Implementations

impl ColumnParallelLinear

pub fn forward(&self, x: &Tensor) -> Tensor

Forward: Y_t = X @ W_t (+ bias). No collective in forward.

pub fn allreduce_grad_x(&self, grad_x: &Tensor) -> Tensor

AllReduce grad_X after backward (called explicitly by the user/executor).