Struct ModelPlacement 

pub struct ModelPlacement {
    pub size_vram_bytes: u64,
    pub total_bytes: u64,
    pub suggested_num_ctx: Option<u32>,
}

Where a loaded model actually sits in memory, from a post-turn probe (Ollama /api/ps). total_bytes is weights + KV + buffers; size_vram_bytes is the part resident in VRAM. size_vram_bytes < total_bytes means the model spilled to CPU/RAM (partial offload → slow). Only Ollama reports this.

Fields

size_vram_bytes: u64

total_bytes: u64

suggested_num_ctx: Option<u32>

Auto-converge target: when the model spilled, the largest num_ctx that would fit instead — or None if it already fits or shrinking can’t help (weights-bound). Computed against the measured footprint.

Trait Implementations

impl Clone for ModelPlacement

fn clone(&self) -> ModelPlacement

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for ModelPlacement

impl Debug for ModelPlacement

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.