Module stream_ordered_alloc

Expand description

Stream-ordered memory allocation (CUDA 11.2+ / 12.x+).

Stream-ordered memory allocation allows memory operations (alloc / free) to participate in the stream execution order, eliminating the need for explicit synchronisation between allocation and kernel launch.

This module provides:

StreamMemoryPool — a memory pool bound to a specific device.
StreamAllocation — a handle to a stream-ordered allocation.
StreamOrderedAllocConfig — pool configuration (sizes, thresholds).
PoolAttribute / PoolUsageStats — attribute queries and statistics.
PoolExportDescriptor / ShareableHandleType — IPC sharing metadata.
stream_alloc / stream_free — convenience free functions.

§The stream-ordered model

Independently of the GPU driver, every pool carries a faithful CPU simulation of the stream-ordered allocator (see stream_ordered_model). Allocations (alloc_async / alloc_on) and frees (free_async / free_on) are sequenced per stream; freed blocks return to the pool once their stream reaches the free point and are then reused by a later same-or-larger request. This makes the allocator’s semantics — visibility ordering, block reuse, and reserved-vs-used accounting — observable on a plain CPU, with no GPU required.

§Platform behaviour

On platforms with a real CUDA driver (Linux, Windows), StreamMemoryPool::new additionally creates a driver-side pool via cuMemPoolCreate. The lower-level cuMem*Async bindings remain available for real-GPU use. StreamMemoryPool::cpu_pool builds a pool backed only by the CPU model, so the full stream-ordered API can be exercised without a driver on any platform.

§Example

use oxicuda_driver::stream_ordered_alloc::*;
use oxicuda_driver::StreamOrderId;

// A pool backed by the faithful CPU model — no GPU driver required.
let config = StreamOrderedAllocConfig::default_for_device(0);
let mut pool = StreamMemoryPool::cpu_pool(config)?;

// A genuine stream-ordering identity.  In a GPU program this is derived
// from a real `Stream` via `StreamMemoryPool::stream_id`; the model
// sequences this token exactly like a real stream would.
let stream = StreamOrderId::from(1);

let mut alloc = pool.alloc_on(1024, stream)?;
assert_eq!(alloc.size(), 1024);
assert!(!alloc.is_freed());

pool.free_on(&mut alloc, stream)?;
assert!(alloc.is_freed());

Structs§

PoolExportDescriptor: Descriptor for exporting a pool for IPC sharing.
PoolUsageStats: Snapshot of pool memory usage.
StreamAllocation: Handle to a stream-ordered memory allocation.
StreamMemoryPool: A memory pool for stream-ordered allocations.
StreamOrderedAllocConfig: Configuration for a stream-ordered memory pool.

Enums§

PoolAttribute: Attributes that can be queried or set on a StreamMemoryPool.
ShareableHandleType: Handle type used for IPC sharing of memory pools.

Constants§

CU_MEMPOOL_ATTR_RELEASE_THRESHOLD: Release threshold in bytes (memory returned to OS when usage drops below).
CU_MEMPOOL_ATTR_RESERVED_MEM_CURRENT: Current reserved memory (bytes) — read-only.
CU_MEMPOOL_ATTR_RESERVED_MEM_HIGH: High-water mark of reserved memory (bytes) — resettable.
CU_MEMPOOL_ATTR_REUSE_ALLOW_INTERNAL_DEPENDENCIES: Pool reuse policy: allow internal dependency insertion.
CU_MEMPOOL_ATTR_REUSE_ALLOW_OPPORTUNISTIC: Pool reuse policy: allow opportunistic reuse.
CU_MEMPOOL_ATTR_REUSE_FOLLOW_EVENT_DEPENDENCIES: Pool reuse policy: follow event dependencies.
CU_MEMPOOL_ATTR_USED_MEM_CURRENT: Current used memory (bytes) — read-only.
CU_MEMPOOL_ATTR_USED_MEM_HIGH: High-water mark of used memory (bytes) — resettable.

Functions§

stream_alloc: Allocate memory on a stream using the default pool for device 0.
stream_free: Free a stream-ordered allocation.