Struct MemoryBreakdownEntry 

pub struct MemoryBreakdownEntry {
    pub buft_name: String,
    pub model: usize,
    pub context: usize,
    pub compute: usize,
}

Memory attributed to a single backend buffer type (e.g. CUDA0, Host).

Fields

buft_name: String

Human-readable buffer-type name from ggml.

model: usize

Bytes used by model weights on this buffer type.

context: usize

Bytes used by the KV / recurrent context cache.

compute: usize

Bytes used by temporary compute buffers.

Implementations

impl MemoryBreakdownEntry

pub fn total(&self) -> usize

Sum of model, context, and compute bytes.

Trait Implementations

impl Clone for MemoryBreakdownEntry

fn clone(&self) -> MemoryBreakdownEntry

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MemoryBreakdownEntry

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for MemoryBreakdownEntry

impl PartialEq for MemoryBreakdownEntry

fn eq(&self, other: &MemoryBreakdownEntry) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for MemoryBreakdownEntry