Enum ShardingStrategy 

pub enum ShardingStrategy {
    FullShard,
    ShardGradOp,
    NoShard,
    HybridShard,
}

Strategy for sharding parameters in FSDP.

Variants

FullShard

Shard parameters, gradients, and optimizer state (ZeRO-3)

ShardGradOp

Shard gradients and optimizer state only (ZeRO-2)

NoShard

No sharding, replicate across ranks (DDP-like)

HybridShard

Hybrid sharding within node, replicate across nodes

Trait Implementations

impl Clone for ShardingStrategy

fn clone(&self) -> ShardingStrategy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ShardingStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ShardingStrategy

fn default() -> Self

Returns the “default value” for a type.

impl PartialEq for ShardingStrategy

fn eq(&self, other: &ShardingStrategy) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for ShardingStrategy

impl Eq for ShardingStrategy

impl StructuralPartialEq for ShardingStrategy