Struct MemoryHook 

pub struct MemoryHook {
    pub name: String,
    pub estimated_free_bytes: usize,
    pub execution_overhead_bytes: usize,
    pub priority: u32,
    pub callback: Box<dyn Fn() -> usize + Send + Sync>,
}

A hook that can free memory or reduce memory demand before an OOM.

Hooks declare upfront how much memory they expect to free, so the memory guard can decide which hooks to call and in what order.

Example: halving a batch when memory is tight

use ferrotorch_gpu::memory_guard::MemoryHook;

let hook = MemoryHook {
    name: "halve_batch_size".into(),
    estimated_free_bytes: 512 * 1024 * 1024, // expects to free ~512 MiB
    execution_overhead_bytes: 4096,           // metadata setup cost
    priority: 10,
    callback: Box::new(|| {
        // ... split the batch, free old tensors ...
        512 * 1024 * 1024 // actual bytes freed
    }),
};

Fields

name: String

Human-readable name (e.g., "halve_batch_size", "free_kv_cache").

estimated_free_bytes: usize

Estimated bytes this hook will free when called.

execution_overhead_bytes: usize

Extra bytes this hook needs temporarily to execute (e.g., metadata setup for a batch split). If the available headroom is less than this overhead the hook is skipped.

priority: u32

Priority: lower values fire first. Hooks at the same priority are called in registration order.

callback: Box<dyn Fn() -> usize + Send + Sync>

The callback. Returns the actual bytes freed (may differ from the estimate).

Trait Implementations

impl Debug for MemoryHook

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.