Enum WrappingPolicy 

pub enum WrappingPolicy {
    WholeProgramWrap,
    TransformerLayerWrap {
        min_params: usize,
    },
    SizeBasedWrap {
        min_num_params: usize,
    },
}

FSDP wrapping strategy

Variants

WholeProgramWrap

Wrap entire model as one FSDP unit

TransformerLayerWrap

Wrap each transformer layer separately (recommended)

Fields

min_params: usize

Minimum number of parameters to wrap a layer

SizeBasedWrap

Wrap by module size threshold

Fields

min_num_params: usize

Minimum number of parameters to trigger wrapping

Trait Implementations

impl Clone for WrappingPolicy

fn clone(&self) -> WrappingPolicy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for WrappingPolicy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for WrappingPolicy

fn eq(&self, other: &WrappingPolicy) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for WrappingPolicy