Skip to main content

Module host_memory

Module host_memory 

Source
Expand description

How much memory the host actually has free, and whether a model fits.

Lives here rather than in a binary because both ferrox-cli and ferrox-server need the same answer, and two copies of a platform probe drift.

Enums§

FitPlan
What to do about a model whose weights may not fit.

Functions§

available_bytes
Bytes a new allocation can reasonably expect to get.
parse_vm_stat
Free plus inactive pages from vm_stat output, in bytes.
plan_for
Decide whether streaming is needed, given what the model weighs and what the host has.