Expand description
How much memory the host actually has free, and whether a model fits.
Lives here rather than in a binary because both ferrox-cli and
ferrox-server need the same answer, and two copies of a
platform probe drift.
Enums§
- FitPlan
- What to do about a model whose weights may not fit.
Functions§
- available_
bytes - Bytes a new allocation can reasonably expect to get.
- parse_
vm_ stat - Free plus inactive pages from
vm_statoutput, in bytes. - plan_
for - Decide whether streaming is needed, given what the model weighs and what the host has.