Enum ShardingStrategy 

pub enum ShardingStrategy {
    Replicated,
    DataParallel,
    ModelParallel,
    DimSharded(usize),
    Pipeline,
    Hybrid,
}

Tensor sharding strategies

Different ways to partition a tensor across multiple devices.

Variants

Replicated

Replicate the full tensor on each device (data parallelism)

DataParallel

Shard along the batch dimension

ModelParallel

Shard along the model dimension (tensor parallelism)

DimSharded(usize)

Shard along a specific dimension

Pipeline

Pipeline parallelism (different layers on different devices)

Hybrid

Combination of strategies

Trait Implementations

impl Clone for ShardingStrategy

fn clone(&self) -> ShardingStrategy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ShardingStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for ShardingStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for ShardingStrategy

fn eq(&self, other: &ShardingStrategy) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for ShardingStrategy

impl Eq for ShardingStrategy

impl StructuralPartialEq for ShardingStrategy