pub fn net_cache_budget_bytes(
memory_ratio: f64,
baseline_free_bytes: u64,
weights_bytes: u64,
fixed_cache_bytes: u64,
) -> i64Expand description
The VRAM a rebuild may spend: a fraction of what was free before the weights loaded, minus the weights, minus whatever the pools need unconditionally.
The fraction is not padding for its own sake – it is the room the
activations, the workspace, and the captured graphs occupy, none of
which is in weights_bytes. Spending it produces an allocation
failure at the first long prompt rather than at startup.
Signed on purpose: an over-committed deployment gets a negative budget, which every consumer below then refuses, rather than a wrapped-around enormous one.