Module async_dispatch 

Async GPU dispatch & back-pressure.

After the launch host function records a kernel launch (via a CUDA Event on real hardware), Wasmtime suspends the calling Wasm fiber by awaiting a DispatchFuture. The runtime is free to schedule other Wasm instances in the meantime. The future resolves when the CUDA Event synchronises, signaling kernel completion.

On no-CUDA hosts the future resolves immediately (it represents work that “ran” only nominally), but the back-pressure machinery still applies — useful for unit-testing the rate-limit logic.

Structs

BackPressure

A back-pressure semaphore plus a live-counter for observability.

BorrowedDispatchPermit

Borrowed counterpart to DispatchPermit returned by BackPressure::acquire_borrowed. Its lifetime is bound to the &BackPressure it was acquired from, so it cannot cross a tokio::spawn boundary — use DispatchPermit for that. In return, acquisition skips the Arc<Semaphore> clone the owned variant pays.

DispatchFuture

A future representing an in-flight GPU dispatch.

DispatchPermit

RAII permit returned by BackPressure::acquire. Dropping it releases the underlying semaphore slot and decrements the live counter.

Enums

BackPressureError

Errors returned by the deadline-aware back-pressure acquire path.

Constants

DEADLINE_NEAR_WINDOW

Window before the configured deadline during which the back-pressure path tightens: new acquires are rejected with BackPressureError::DeadlineNear so the in-flight cohort can drain without being further saturated by fresh launches. Picked at 50 ms — five default epoch ticks — which empirically lets a per-tile loop wind down (typical tile completion ≤ a few ms) without surrendering useful budget on the common path.

DEFAULT_MAX_CONCURRENT_GPU_OPS

Default maximum number of concurrent GPU operations across the process. Mirrors the plan’s choice of “a few times the number of SMs” — tuned at startup to match the deployed hardware in S17.