Module stream_ordered_model 

Faithful CPU model of CUDA stream-ordered memory allocation.

This module implements the semantics of the CUDA stream-ordered allocator (cuMemAllocAsync / cuMemFreeAsync / the cuMemPool* family) without any GPU. It is a self-contained, deterministic simulation that the StreamMemoryPool drives on every platform so that the allocator’s behaviour can be exercised and reasoned about on a plain CPU.

What “stream-ordered” means

In CUDA, cuMemAllocAsync(ptr, size, stream) returns a pointer immediately to the host, but the memory is only valid on the GPU once the stream reaches the allocation point. Symmetrically, cuMemFreeAsync(ptr, stream) records a free that only takes effect once the stream reaches the free point — until then the memory is still in use by earlier work on that stream. A pointer freed on stream A may be reused by a later allocation, and that reuse is only safe with respect to work on another stream B if B has been ordered after the free (e.g. via an event).

The model

Each stream is modelled by a StreamClock: a monotonically increasing submit counter (the position at which the next operation is enqueued) and a reached counter (how far the stream has actually executed). Submitting an operation returns its sequence number; the operation is complete once reached >= seq. Advancing a stream (the model’s analogue of cuStreamSynchronize) sets reached = submit, retiring every pending operation in FIFO order — exactly the in-order guarantee CUDA gives.

Memory is modelled by a flat virtual address space. Live allocations own a Block; a stream-ordered free moves the block onto a pending-free queue tagged with the freeing stream and the free’s sequence number. When that stream reaches the free point the block is returned to the pool’s free list and becomes eligible for reuse by a later same-or-larger request (first-fit over the free list, preferring exact size). reserved bytes count everything the pool has carved from the (virtual) device, whereas used bytes count only currently-live allocations; trimming releases free-list bytes above the release threshold back to the device.

Structs

ModelAllocation

Record of a live allocation produced by the model.

ModelLimits

Configuration knobs the model needs from the pool.

StreamOrderId

Identifier of a stream within the model.

StreamOrderModel

The stream-ordered allocation engine.