Struct MemoryWatchdog 

pub struct MemoryWatchdog { /* private fields */ }

Background monitor that pauses training when free VRAM drops below a threshold.

Create a watchdog, wrap it in an Arc, and call start to spawn the monitoring thread. Between training batches, call wait_if_paused to block until memory pressure is resolved.

use std::sync::Arc;
use std::time::Duration;
use ferrotorch_gpu::memory_guard::MemoryWatchdog;
use ferrotorch_gpu::GpuDevice;

let device = Arc::new(GpuDevice::new(0).unwrap());
let watchdog = Arc::new(MemoryWatchdog::new(
    device,
    512 * 1024 * 1024, // pause when <512 MiB free
    Duration::from_secs(1),
));
let handle = Arc::clone(&watchdog).start();

// In training loop:
watchdog.wait_if_paused();

Implementations

impl MemoryWatchdog

pub fn new( device: Arc<GpuDevice>, pressure_threshold_bytes: usize, check_interval: Duration, ) -> Self

Create a new watchdog. Does not start monitoring until start is called.

pub fn start(self: Arc<Self>) -> JoinHandle<()>

Start the monitoring thread. Returns a JoinHandle that can be used to wait for shutdown (after calling stop).

pub fn stop(&self)

Signal the background thread to exit.

pub fn check_pressure(&self) -> bool

Returns true if the watchdog currently has training paused due to memory pressure.

pub fn wait_if_paused(&self)

Block the calling thread until memory pressure is resolved. Call this between training batches.

pub fn wait_for_first_check(&self, timeout: Duration)

Block until the watchdog has completed at least one check cycle. Useful in tests to avoid timing races.

pub fn pressure_threshold_bytes(&self) -> usize

The pressure threshold in bytes.

Trait Implementations

impl Debug for MemoryWatchdog

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.