Module stream_ordered_alloc

Expand description

Stream-ordered memory allocation (CUDA 11.2+ / 12.x+).

Stream-ordered memory allocation allows memory operations (alloc / free) to participate in the stream execution order, eliminating the need for explicit synchronisation between allocation and kernel launch.

This module provides:

StreamMemoryPool — a memory pool bound to a specific device.
StreamAllocation — a handle to a stream-ordered allocation.
StreamOrderedAllocConfig — pool configuration (sizes, thresholds).
PoolAttribute / PoolUsageStats — attribute queries and statistics.
PoolExportDescriptor / ShareableHandleType — IPC sharing metadata.
stream_alloc / stream_free — convenience free functions.

§Platform behaviour

On macOS (where NVIDIA dropped CUDA support), all operations that would require the GPU driver return Err(CudaError::NotSupported). Config validation, statistics tracking, and accessor methods work everywhere.

§Example

use oxicuda_driver::stream_ordered_alloc::*;

let config = StreamOrderedAllocConfig::default_for_device(0);
let mut pool = StreamMemoryPool::new(config)?;

let stream_handle = 0u64; // placeholder
let mut alloc = pool.alloc_async(1024, stream_handle)?;
assert_eq!(alloc.size(), 1024);
assert!(!alloc.is_freed());

pool.free_async(&mut alloc)?;
assert!(alloc.is_freed());

Structs§

PoolExportDescriptor: Descriptor for exporting a pool for IPC sharing.
PoolUsageStats: Snapshot of pool memory usage.
StreamAllocation: Handle to a stream-ordered memory allocation.
StreamMemoryPool: A memory pool for stream-ordered allocations.
StreamOrderedAllocConfig: Configuration for a stream-ordered memory pool.

Enums§

PoolAttribute: Attributes that can be queried or set on a StreamMemoryPool.
ShareableHandleType: Handle type used for IPC sharing of memory pools.

Constants§

CU_MEMPOOL_ATTR_RELEASE_THRESHOLD: Release threshold in bytes (memory returned to OS when usage drops below).
CU_MEMPOOL_ATTR_RESERVED_MEM_CURRENT: Current reserved memory (bytes) — read-only.
CU_MEMPOOL_ATTR_RESERVED_MEM_HIGH: High-water mark of reserved memory (bytes) — resettable.
CU_MEMPOOL_ATTR_REUSE_ALLOW_INTERNAL_DEPENDENCIES: Pool reuse policy: allow internal dependency insertion.
CU_MEMPOOL_ATTR_REUSE_ALLOW_OPPORTUNISTIC: Pool reuse policy: allow opportunistic reuse.
CU_MEMPOOL_ATTR_REUSE_FOLLOW_EVENT_DEPENDENCIES: Pool reuse policy: follow event dependencies.
CU_MEMPOOL_ATTR_USED_MEM_CURRENT: Current used memory (bytes) — read-only.
CU_MEMPOOL_ATTR_USED_MEM_HIGH: High-water mark of used memory (bytes) — resettable.

Functions§

stream_alloc: Allocate memory on a stream using the default pool for device 0.
stream_free: Free a stream-ordered allocation using a temporary default pool.