Skip to main content

Crate oxidelake_memory

Crate oxidelake_memory 

Source
Expand description

Memory management for OxideLake (docs/SPEC.md §2.1): aligned host buffers, pinned (CUDA) and unified (Metal) allocations, Arrow IPC encoding for spill files, and the three-tier spill manager.

  • AlignedBuf — zeroed, explicitly aligned host allocation.
  • HostAlloc — host buffer that is pinned when a PinnedProvider can deliver it and pageable otherwise; the class is observable.
  • SpillManager — device → host → disk demotion with LRU eviction; the disk tier is Arrow IPC written through an object_store::ObjectStore.

Dependency direction: depends on oxidelake-core only.

Re-exports§

pub use aligned::AlignedBuf;
pub use class::MemoryClass;
pub use host::HostAlloc;
pub use host::PinnedProvider;
pub use spill::BatchId;
pub use spill::DeviceResident;
pub use spill::SpillBudget;
pub use spill::SpillManager;
pub use spill::Tier;

Modules§

aligned
Owned, zero-initialised host allocations with explicit alignment.
class
Host memory classes.
host
Host buffers: pinned when the backend can provide it, pageable otherwise.
ipc
Arrow IPC encoding for spill and cache files: raw Arrow buffers with 64-byte alignment and no compression, so batches come back without decode.
pinned
Page-locked host memory through the CUDA driver (feature cuda).
spill
Three-tier spill manager: device VRAM → host RAM → local disk.