Enum LayerWeightResidency 

#[non_exhaustive]
pub enum LayerWeightResidency {
    FullyResident,
    LayerwiseHost(LayerwiseLoadOptions),
    DenseDiskStream(DenseDiskStreamLoadOptions),
}

Placement policy for ordinary architecture execution units.

Variants (Non-exhaustive)

Non-exhaustive enums could have additional variants added in future. Therefore, when matching against variants of non-exhaustive enums, an extra wildcard arm must be added to account for any future variants.

FullyResident

Construct every rank-local module once and retain it on the execution device.

LayerwiseHost(LayerwiseLoadOptions)

Eagerly materialize units on a host stream and use a bounded device window.

DenseDiskStream(DenseDiskStreamLoadOptions)

Leave units cold on disk and use finite host and device caches.

Implementations

impl LayerWeightResidency

pub const fn max_cached_shards(self) -> usize

Returns the checkpoint shard/reader cache bound carried by this policy.

pub const fn sample_backend_memory(self) -> bool

Returns whether backend allocator memory should be sampled.

pub const fn sample_process_memory(self) -> bool

Returns whether process memory should be sampled.

pub fn device_depth(self, unit_count: usize) -> usize

Returns the maximum execution-device unit window.

pub fn offload(self) -> Result<OffloadConfig, WeightResidencyPolicyError>

Resolves this policy into the shared offload configuration.

pub const fn dense(self) -> Option<DenseDiskStreamLoadOptions>

Returns dense-stream controls when this policy streams from disk.

pub const fn is_fully_resident(self) -> bool

Returns whether every ordinary execution unit remains device-resident.

pub const fn execution_residency(self) -> ExecutionResidency

Returns the stable physical execution classification.

Trait Implementations

impl Clone for LayerWeightResidency

fn clone(&self) -> LayerWeightResidency

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for LayerWeightResidency

impl Debug for LayerWeightResidency

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LayerWeightResidency

fn default() -> LayerWeightResidency

Returns the “default value” for a type.

impl Eq for LayerWeightResidency

impl From<DenseDiskStreamLoadOptions> for LayerWeightResidency

fn from(value: DenseDiskStreamLoadOptions) -> Self

Converts to this type from the input type.

impl From<LayerwiseLoadOptions> for LayerWeightResidency

fn from(value: LayerwiseLoadOptions) -> Self

Converts to this type from the input type.

impl From<OrdinaryWeightResidency> for LayerWeightResidency

fn from(value: OrdinaryWeightResidency) -> Self

Converts to this type from the input type.

impl PartialEq for LayerWeightResidency

fn eq(&self, other: &LayerWeightResidency) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for LayerWeightResidency