Struct VirtualMemoryPool 

pub struct VirtualMemoryPool { /* private fields */ }

Elastic memory pool with virtual memory backing.

This pool reserves a large virtual address space but only allocates physical memory on-demand when allocate() is called. This enables:

• Large virtual capacity (e.g., 128GB) with minimal initial physical usage
• Dynamic allocation/deallocation based on workload
• Reduced memory waste for bursty workloads

Example

use candle_cuda_vmm::VirtualMemoryPool;
use candle_core::Device;

let device = Device::new_cuda(0)?;
let mut pool = VirtualMemoryPool::new(
    128 * 1024 * 1024 * 1024, // 128GB virtual capacity
    2 * 1024 * 1024,          // 2MB page size
    device,
)?;

// Allocate 1GB of physical memory on-demand
let addr = pool.allocate(0, 1024 * 1024 * 1024)?;
println!("Physical usage: {} bytes", pool.physical_memory_usage());

// Deallocate when done
pool.deallocate(0, 1024 * 1024 * 1024)?;

Implementations

impl VirtualMemoryPool

pub fn new(capacity: usize, page_size: usize, device: Device) -> Result<Self>

Create a new virtual memory pool.

Arguments

• capacity - Maximum virtual address space (e.g., 128GB).
• page_size - Page granularity (e.g., 2MB for large pages).
• device - CUDA device.

Returns

Pool with reserved virtual address space, no physical memory allocated.

Errors

Returns error if:

• Device is not a CUDA device
• Page size is invalid (not power of 2 or < 64KB)
• Virtual address reservation fails

pub fn allocate(&mut self, offset: usize, size: usize) -> Result<usize>

Allocate and map physical pages on-demand.

Arguments

• offset - Offset in virtual address space (bytes).
• size - Number of bytes to allocate.

Returns

Base virtual address of allocated region.

Errors

Returns error if:

• Offset/size out of bounds
• Region already allocated
• Physical memory allocation fails

pub fn deallocate(&mut self, offset: usize, size: usize) -> Result<()>

Unmap and free physical pages.

Arguments

• offset - Offset in virtual address space (bytes).
• size - Number of bytes to free.

Errors

Returns error if:

• Offset/size out of bounds
• Region not allocated

pub fn physical_memory_usage(&self) -> usize

Get current physical memory usage in bytes.

pub fn capacity(&self) -> usize

Get virtual address space capacity in bytes.

pub fn base_address(&self) -> usize

Get base virtual address.

pub fn page_size(&self) -> usize

Get page size in bytes.

pub fn is_mapped(&self, offset: usize, size: usize) -> bool

Check if a range is currently mapped.

Arguments

• offset - Offset in virtual address space (bytes).
• size - Size to check (bytes).

Returns

True if entire range is mapped, false otherwise.

pub fn compact(&mut self) -> Result<()>

Compact pool by coalescing free pages (no-op for now, future optimization).

pub fn stats(&self) -> MemoryStats

Get memory statistics.