Skip to main content

Module residency

Module residency 

Source
Expand description

Unified-memory budget and co-residency (card mesh-placement-20; ADR-101 section 6). One ledger is shared by every managed adapter on a node: an instance reserves its weights plus KV budget when it starts and releases them when it stops or is unloaded. A start that would exceed the budget, or that excludes (or is excluded by) a running role, is refused as unplaceable, with the reason.

Memory is counted in decimal gigabytes at the edges (ram_gb of the model lab’s roster, the operator’s budget) and in bytes here.

Structs§

ResidencyLedger
The node’s resident instances.

Enums§

CoResidency
Why an instance cannot be resident beside the others.

Constants§

GB
Bytes in the roster’s and the operator’s “GB”.

Functions§

footprint
Bytes a spec holds while resident.