Expand description
Memory management module.
Memory pools and the bookkeeping behind a Handle.
The value types a client sees — reports, configuration, the handle itself —
are cubecl-runtime’s and re-exported here.
Modules§
- drop_
queue - Export utilities to keep track of CPU buffers when performing async data copies.
- relocation
- Moving live allocations off outdated pages.
Structs§
- Auxiliary
Memory Report - A memory a runtime keeps for its own use, and what it holds.
- Error
Graph - Every failure the device is still holding, and how many allocations carry each one.
- Failure
Id - The id a tainted allocation carries, naming the failure that left its bytes unwritten.
- Managed
Memory Binding - Binding of a memory handle
- Managed
Memory Handle - Managed Memory handle
- Managed
Memory Id - Managed memory unique identifier.
- Memory
Management - Reserves and keeps track of chunks of memory in the storage, and slices upon these chunks.
- Memory
Management Options - The options for creating a new
MemoryManagementinstance. - Memory
Pool Report - A structured snapshot of one memory pool: its shape, its current usage, and the high-water marks a memory plan is derived from.
- Memory
Pools Report - What a memory holds, pool by pool.
- Memory
Record - A
MemoryReportas the environment records it: a snapshot of every stream’s memory on the device at a moment the caller named, written byClient::record_memory. - Memory
Report - Everything the memory of a
MemoryScopeholds, stream by stream: the single place memory is read from. A caller that wants the totals asks for theusage. - Memory
Usage - Amount of memory in use by this allocator and statistics on how much memory is reserved and wasted in total.
- Page
Guard - Keeps a page as it is for as long as it lives: nothing new is carved from it, it is never released, and relocation leaves everything on it in place.
- Shared
Memory Bindings - A list of bindings that are shared across multiple streams.
- Skipped
- One launch a failure stopped: the kernel that did not run, the buffer whose claim stopped it, and what it would have produced.
- Stream
Memory Report - What one stream’s memory holds.
- Taint
- The tainted regions of one allocation, refcounted into the device’s
ErrorGraph. - Weak
Memory Binding - A
ManagedMemoryBindingthat does not keep its memory reserved: what a record of an allocation holds when holding the allocation itself would change what the pools may reuse.
Enums§
- Claim
- Why the bytes of one buffer cannot be trusted, as a read finds it.
- Cleanup
- Why a cleanup runs, which decides how much it gives back.
- Memory
Allocation Mode - The mode of allocation used.
- Memory
Configuration - High level configuration of memory management.
- Memory
Pool Kind - The pool shape a
MemoryPoolReportdescribes, carrying the pool’s effective configuration (after alignment rounding and page-size shrinking). - Memory
Scope - Which memory a
MemoryReportcovers. - Page
Update - What a reservation may do to the pages the memory holds to find room.
Constants§
- EXCLUSIVE_
MEMORY_ ONLY - Whether this build refuses pools that share a page between allocations —
the
exclusive-memory-onlyfeature, or a wasm target.
Traits§
- Memory
Handle - The managed tensor buffer handle that points to some memory segment. It should not contain actual data.
Functions§
- optimal_
align - Calculates a best-effort heuristic for the alignment of row-aligned tensors.
Prefers contiguous alignments for unit dimensions, 16-byte minimum alignment for non-unit,
scaling with input size up to
buffer_align.