Struct RowParallelLinear 

pub struct RowParallelLinear { /* private fields */ }

Row-parallel linear layer.

Splits the weight matrix along the row dimension across ranks. Each rank has a portion of the input features.

Implementations

impl RowParallelLinear

pub fn new( in_features: usize, out_features: usize, bias: bool, process_group: ProcessGroup, input_is_parallel: bool, ) -> Self

Creates a new row-parallel linear layer.

Trait Implementations

impl Module for RowParallelLinear

fn forward(&self, input: &Variable) -> Variable

Performs the forward pass.

fn parameters(&self) -> Vec<Parameter>

Returns all parameters of this module.

fn named_parameters(&self) -> HashMap<String, Parameter>

Returns named parameters of this module.

fn num_parameters(&self) -> usize

Returns the number of trainable parameters.

fn train(&mut self)

Sets the module to training mode.

fn eval(&mut self)

Sets the module to evaluation mode.

fn set_training(&mut self, _training: bool)

Sets the training mode.

fn is_training(&self) -> bool

Returns whether the module is in training mode.

fn zero_grad(&self)

Zeros all gradients of parameters.

fn name(&self) -> &'static str

Returns the module name for debugging.