Expand description
Memory management value types: reports, configuration and the managed
handle. The pools behind them are cubecl-server’s.
Structs§
- Auxiliary
Memory Report - A memory a runtime keeps for its own use, and what it holds.
- Managed
Memory Binding - Binding of a memory handle
- Managed
Memory Handle - Managed Memory handle
- Managed
Memory Id - Managed memory unique identifier.
- Memory
Pool Report - A structured snapshot of one memory pool: its shape, its current usage, and the high-water marks a memory plan is derived from.
- Memory
Pools Report - What a memory holds, pool by pool.
- Memory
Record - A
MemoryReportas the environment records it: a snapshot of every stream’s memory on the device at a moment the caller named, written byClient::record_memory. - Memory
Report - Everything the memory of a
MemoryScopeholds, stream by stream: the single place memory is read from. A caller that wants the totals asks for theusage. - Memory
Usage - Amount of memory in use by this allocator and statistics on how much memory is reserved and wasted in total.
- Page
Guard - Keeps a page as it is for as long as it lives: nothing new is carved from it, it is never released, and relocation leaves everything on it in place.
- Shared
Memory Bindings - A list of bindings that are shared across multiple streams.
- Stream
Memory Report - What one stream’s memory holds.
- Weak
Memory Binding - A
ManagedMemoryBindingthat does not keep its memory reserved: what a record of an allocation holds when holding the allocation itself would change what the pools may reuse.
Enums§
- Memory
Allocation Mode - The mode of allocation used.
- Memory
Configuration - High level configuration of memory management.
- Memory
Pool Kind - The pool shape a
MemoryPoolReportdescribes, carrying the pool’s effective configuration (after alignment rounding and page-size shrinking). - Memory
Scope - Which memory a
MemoryReportcovers.
Constants§
- EXCLUSIVE_
MEMORY_ ONLY - Whether this build refuses pools that share a page between allocations —
the
exclusive-memory-onlyfeature, or a wasm target.
Traits§
- Memory
Handle - The managed tensor buffer handle that points to some memory segment. It should not contain actual data.
Functions§
- optimal_
align - Calculates a best-effort heuristic for the alignment of row-aligned tensors.
Prefers contiguous alignments for unit dimensions, 16-byte minimum alignment for non-unit,
scaling with input size up to
buffer_align.