Enum ShardingStrategy 

pub enum ShardingStrategy {
    FullShard,
    ShardGradOp,
    NoShard,
    HybridShard {
        num_model_replicas: usize,
    },
}

Sharding strategy for FSDP — analogous to ZeRO stages.

Variants

FullShard

ZeRO-3: shard params + grads + opt state (maximum memory savings).

ShardGradOp

ZeRO-2: shard grads + opt state only; parameters are replicated.

NoShard

DDP: replicate everything — no sharding.

HybridShard

ZeRO-3 within a node, replicate model across nodes.

Fields

num_model_replicas: usize

Trait Implementations

impl Clone for ShardingStrategy

fn clone(&self) -> ShardingStrategy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ShardingStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for ShardingStrategy

fn eq(&self, other: &ShardingStrategy) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for ShardingStrategy