Expand description
Memory management module.
Modules§
- drop_
queue - Export utilities to keep track of CPU buffers when performing async data copies.
Structs§
- Managed
Memory Binding - Binding of a memory handle
- Managed
Memory Handle - Managed Memory handle
- Managed
Memory Id - Managed memory unique identifier.
- Memory
Management - Reserves and keeps track of chunks of memory in the storage, and slices upon these chunks.
- Memory
Management Options - The options for creating a new
MemoryManagementinstance. - Memory
Pool Options - Options to create a memory pool.
- Memory
Pool Report - A structured snapshot of one memory pool: its shape, its current usage, and the high-water marks a memory plan is derived from.
- Memory
Report - A per-pool report of one
MemoryManagementinstance — the read side of a measured memory plan. - Memory
Usage - Amount of memory in use by this allocator and statistics on how much memory is reserved and wasted in total.
- Shared
Memory Bindings - A list of bindings that are shared across multiple streams.
Enums§
- Install
Memory Pools Error - Why installing a dynamic pool layout did not take effect.
- Memory
Allocation Mode - The mode of allocation used.
- Memory
Configuration - High level configuration of memory management.
- Memory
Pool Kind - The pool shape a
MemoryPoolReportdescribes, carrying the pool’s effective configuration (after alignment rounding and page-size shrinking). - Pool
Config Error - Why a
memory.poolsconfig could not be turned into a pool layout. - Pool
Type - The type of memory pool to use.
Traits§
- Memory
Handle - The managed tensor buffer handle that points to some memory segment. It should not contain actual data.
Functions§
- optimal_
align - Calculates a best-effort heuristic for the alignment of row-aligned tensors.
Prefers contiguous alignments for unit dimensions, 16-byte minimum alignment for non-unit,
scaling with input size up to
buffer_align.