Enum WeightResidency 

#[non_exhaustive]
pub enum WeightResidency {
    Layers(LayerWeightResidency),
    #[non_exhaustive]
    IndependentParameterBanks {
        ordinary: OrdinaryWeightResidency,
        cache: ParameterBankLoadOptions,
    },
}

Composable ordinary-unit and independently addressable bank placement.

Variants (Non-exhaustive)

Non-exhaustive enums could have additional variants added in future. Therefore, when matching against variants of non-exhaustive enums, an extra wildcard arm must be added to account for any future variants.

Layers(LayerWeightResidency)

All parameters share the ordinary unit residency allocation.

#[non_exhaustive]

IndependentParameterBanks

Addressable bank members are independent units beside bounded ordinary units.

Fields

Non-exhaustive enum variants could have additional fields added in future. Therefore, non-exhaustive enum variants cannot be constructed in external crates and cannot be matched against.

ordinary: OrdinaryWeightResidency

Bounded ordinary-unit policy.

cache: ParameterBankLoadOptions

Bank-member-granular cache controls.

Implementations

impl WeightResidency

pub const fn fully_resident() -> Self

Keeps every rank-owned parameter on the execution device.

pub const fn layerwise_host(options: LayerwiseLoadOptions) -> Self

Eagerly materializes ordinary units on host behind a bounded device window.

pub const fn dense_disk_stream(options: DenseDiskStreamLoadOptions) -> Self

Leaves ordinary units cold on disk behind finite host/device caches.

pub const fn with_layers(layers: LayerWeightResidency) -> Self

Keeps every owned parameter in its ordinary unit residency allocation.

pub const fn with_independent_parameter_banks( ordinary: OrdinaryWeightResidency, cache: ParameterBankLoadOptions, ) -> Self

Gives addressable parameter banks an independent cache beside ordinary units.

pub const fn layers(self) -> LayerWeightResidency

Returns ordinary-unit placement.

pub const fn parameter_banks(self) -> ParameterBankResidency

Returns independently addressable bank placement relative to ordinary units.

pub const fn parameter_bank_cache(self) -> Option<ParameterBankLoadOptions>

Returns independently cached parameter-bank controls, when selected.

pub const fn ordinary_residency(self) -> Option<OrdinaryWeightResidency>

Returns ordinary placement paired with an independent parameter-bank cache.

pub const fn ordinary_is_fully_resident(self) -> bool

Returns whether every ordinary parameter remains resident.

pub const fn is_fully_resident(self) -> bool

Returns whether every ordinary unit remains resident.

pub const fn max_cached_shards(self) -> usize

Returns the common checkpoint shard/reader cache bound.

Trait Implementations

impl Clone for WeightResidency

fn clone(&self) -> WeightResidency

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for WeightResidency

impl Debug for WeightResidency

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for WeightResidency

fn default() -> Self

Returns the “default value” for a type.

impl Eq for WeightResidency

impl PartialEq for WeightResidency

fn eq(&self, other: &WeightResidency) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for WeightResidency