Struct MemoryManagement 

pub struct MemoryManagement<Storage> { /* private fields */ }

Reserves and keeps track of chunks of memory in the storage, and slices upon these chunks.

Implementations

impl<Storage: ComputeStorage> MemoryManagement<Storage>

pub fn from_configuration( storage: Storage, properties: &MemoryDeviceProperties, config: MemoryConfiguration, logger: Arc<ServerLogger>, options: MemoryManagementOptions, ) -> Self

Creates the options from device limits.

pub fn install_pools( &mut self, config: MemoryConfiguration, properties: &MemoryDeviceProperties, ) -> Result<(), InstallMemoryPoolsError>

Replace the dynamic pools with ones built from a new layout.

The old pools are cleaned up first (every currently-free page returned to the driver) and then discarded — this installs new pools, it does not re-tune the existing ones. That is why it only happens when no live allocation remains in them: a live slice carries its pool position, so swapping the pool list under it would leave that position pointing at a different pool. The caller installs at a quiescent point (e.g. right after unloading a model), so a refusal is the exceptional path, not the normal one.

Rebuilding resets each pool’s high-water marks, which is what lets a measured plan be read from a pass that follows a rebuild rather than from the process’s whole history.

The persistent pool is untouched: its slices route through a fixed sentinel position and its layout is model-agnostic.

Errors

PoolsInUse when something is still live in the dynamic pools; the old layout is kept and nothing is disturbed. Retry once the work holding them drains.

pub fn capture_begin(&mut self)

Begin a graph capture: force every allocation into the persistent pool — exact-fit slices with no bucket padding, which is what a graph’s static shapes want — and start recording which slices the window hands out (see reserve). Every slice the window touches belongs to the graph at capture_end; anything it never touches (pre-existing live buffers, idle free slices) does not. Slices stay reusable within the window — warmup populates the pool, then the capture run reuses those slices without a fresh device allocation (illegal mid-capture). Sets the mode directly, overriding the config gate that mode honors. If a capture is already active, only the mode is re-forced — the original capture keeps its touched set and restore state.

pub fn capture_priming_end(&mut self)

End the priming phase and release the slices warmup retained, returning them to the pool as free. Call immediately before the capture window opens.

After this the pool holds every slice a warmup pass touched, all of them free, so the recorded run reuses them instead of growing the pool (see [CaptureState::primed]). No-op when no capture is active or priming already ended.

pub fn capture_end(&mut self) -> Vec<ManagedMemoryHandle>

End a graph capture: restore the previous allocation mode and return a retained handle to every persistent slice the window touched — exactly the memory the graph’s recorded kernels replay against. The caller pins these on the graph so the pool never reuses graph memory (which a replay would corrupt); dropping the graph drops the handles and releases the slices. Slices the window never touched are left alone, so a pre-existing live buffer keeps its reuse and in-place (can_mut) semantics. Empty if no capture was active.

pub fn mode(&mut self, mode: MemoryAllocationMode)

Change the mode of allocation.

Persistent windows nest: a Persistent call opens one, an Auto call closes one, and the effective mode stays Persistent while any window is open. Callers routinely nest without knowing it — a module load opens a window around the whole load while the parameter machinery underneath opens one per parameter — and without the depth, the first inner window’s exit would flip the rest of the outer window back to Auto: weights landing in the dynamic pools, which then refuse every later rebuild (install_pools) for the model’s whole life.

pub fn cleanup(&mut self, explicit: bool)

Cleanup allocations in pools that are deemed unnecessary.

pub fn get_cursor(&self, binding: ManagedMemoryBinding) -> Result<u64, IoError>

Returns the storage from the specified binding

pub fn get_storage( &mut self, binding: ManagedMemoryBinding, ) -> Result<StorageHandle, IoError>

Returns the storage from the specified binding.

This is the funnel every buffer dereference passes through (get_resource delegates here), so it is where a lazily-carved allocation gets its real device backing: the handle returned always refers to mapped memory.

pub fn get_resource( &mut self, binding: ManagedMemoryBinding, offset_start: Option<u64>, offset_end: Option<u64>, ) -> Result<Storage::Resource, IoError>

Returns the resource from the storage at the specified handle

pub fn reserve(&mut self, size: u64) -> Result<ManagedMemoryHandle, IoError>

Finds a spot in memory for a resource with the given size in bytes, and returns a handle to it

pub fn storage(&mut self) -> &mut Storage

Fetch the storage used by the memory manager.

Notes

The storage should probably not be used for allocations since the handles won’t be compatible with the ones provided by the current trait. Prefer using the alloc and dealloc functions.

This is useful if you need to time the deallocations based on async computation, or to change the mode of storage for different reasons.

pub fn memory_usage(&self) -> MemoryUsage

Get the current memory usage.

pub fn memory_report(&self) -> MemoryReport

A structured per-pool report: each pool’s shape, usage, and high-water marks, in allocation-routing order.

The read side of a measured memory plan — the cycle, and what the marks cover, is on MemoryReport.

pub fn print_memory_usage(&self)

Print out a report of the current memory usage.

pub fn bind( &mut self, reserved: ManagedMemoryHandle, assigned: ManagedMemoryHandle, cursor: u64, ) -> Result<(), IoError>

Binds the given handle to a [MemorySlot].

Trait Implementations

impl<Storage> Debug for MemoryManagement<Storage>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<Storage: ComputeStorage> Display for MemoryManagement<Storage>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.