Module virtual_memory 

Virtual Memory Abstraction for Tiered Storage

This module provides a unified memory interface that spans multiple storage tiers:

• Tier 0 (Hot): GPU VRAM - Fastest, limited capacity
• Tier 1 (Warm): Host RAM - Fast, moderate capacity
• Tier 2 (Cold): SSD/Disk - Slow, large capacity

Design

The virtual memory system uses a handle-based allocation model where data can transparently migrate between tiers based on access patterns. Each allocation has a “home tier” (where data is persisted) and a “current tier” (where data is currently resident for fast access).

┌──────────────────────────────────────────────────────────────┐
│                  Virtual Memory Manager                       │
│  ┌─────────────────┬─────────────────┬─────────────────┐     │
│  │    VRAM Pool    │    Host Pool    │    Disk Pool    │     │
│  │   (Tier 0)      │    (Tier 1)     │    (Tier 2)     │     │
│  └────────┬────────┴────────┬────────┴────────┬────────┘     │
│           │                 │                 │               │
│           ▼                 ▼                 ▼               │
│      GPU Memory       Host Memory        File System         │
└──────────────────────────────────────────────────────────────┘

Features

• Transparent Migration: Data moves between tiers automatically
• LRU Eviction: Least-recently-used data evicted from faster tiers
• Pinning: Prevent specific allocations from being evicted
• Async I/O: Optional async transfers for disk operations

Example

use embeddenator_vsa::virtual_memory::{VirtualMemory, VirtualMemoryConfig, MemoryTier};

let config = VirtualMemoryConfig::default();
let vmem = VirtualMemory::new(config)?;

// Allocate memory (starts in host RAM)
let handle = vmem.allocate(1024 * 1024, MemoryTier::Host)?;

// Write data
vmem.write(&handle, &data)?;

// Promote to VRAM for GPU access
vmem.promote(&handle, MemoryTier::Vram)?;

// Read data (transparent access regardless of tier)
let data = vmem.read(&handle)?;

Structs

VMemHandle

Handle to a virtual memory allocation

VirtualMemory

Virtual memory manager providing unified access to tiered storage

VirtualMemoryConfig

Configuration for virtual memory

VirtualMemoryStats

Statistics about virtual memory usage

Enums

MemoryTier

Memory tier indicating where data resides

VirtualMemoryError

Error type for virtual memory operations