Skip to main content

auto_cpu_layers

ferrox_core::placement

Function auto_cpu_layers 

Source 
pub fn auto_cpu_layers(
    num_layers: usize,
    bank_bytes: u64,
    pin_budget_bytes: Option<u64>,
) -> BTreeSet<u32>
Expand description

The layers to serve on the CPU when the expert banks do not fit the host’s page-locking budget.

None means “no cap applies, or the banks fit”: every layer can be pinned and served from the GPU expert cache. Otherwise enough layers are moved to the CPU that the remaining pinned bytes fit, taken from both ends of the stack for the reason in the module docs.