Expand description
§ironaccelerator-cuda
Low-level, hardware-agnostic CUDA interface. Targets the CUDA Toolkit
13.2 API surface via iron_cuda_sys, with safe wrappers around the
driver, NVRTC, and every major vendor library (cuBLAS / cuBLASLt / cuDNN
/ cuRAND / cuSPARSE / cuSOLVER / cuFFT / cuTENSOR / NCCL / NVTX / CUPTI).
Scope. This crate is a driver substrate. It does not ship kernels, workload planners, FP8 recipes, attention/MoE implementations, or autotuners — those belong to downstream libraries layered on top. What’s here:
drv— Device / Stream / Event / DeviceBuf / Module / Functionkernel— NVRTC compile + in-memory and on-disk PTX cacheblas,cudnn,fft,cusparse,cusolver,cutensor,rng,nccl— per-library handle plumbingadvanced— VMM, green contexts, multicast teams, conditional graph nodesgraph,launch,peer,pinned,streams,events,alloc— driver-level primitivesobserve,profile,cupti— NVTX / profiler hookscudarc_compat— drop-in compatibility surface for cudarc userssys— the raw FFI re-export for callers that need it
§If you want to…
| Task | Use |
|---|---|
| port existing cudarc 0.19 code | cudarc_compat (drop-in use swap) |
| open a device + create a stream | drv::Device::open then drv::Stream::new |
| allocate a typed device buffer | drv::DeviceBuf::alloc / alloc::alloc |
| copy host ↔ device on a stream | drv::DeviceBuf::copy_from_host / copy_to_host |
| compile a CUDA C++ kernel at runtime | kernel::get_or_compile (cached on disk per arch+source) |
| launch a kernel with typed args | drv::Function::launch or launch::launch_1d / _2d |
| record + measure GPU time | events::Timer (or drv::TimingEvent directly) |
| capture & replay a stream as a CUDA graph | drv::Stream::begin_capture → drv::GraphExec |
| call cuBLASLt / cuDNN / cuFFT / NCCL | the matching blas/cudnn/fft/nccl module |
| drop down to raw FFI | sys (re-exports iron_cuda_sys) |
§Performance posture
Every hot path is #[inline]. The driver fns() lookup uses an
AtomicPtr<DriverFns> fast-path cache, and each Device/Stream/Event/
Module/Function caches &'static DriverFns at construction — so a
wrapped op costs one struct-field load + one FFI entry. See the workspace
README for benchmarks against cudarc 0.19; alloc/free is ~2× faster.
Re-exports§
pub use iron_cuda_sys as sys;
Modules§
- advanced
- CUDA 12.3+/12.4+/13.x advanced primitives: virtual memory, multicast, green contexts, graph conditional nodes, confidential-compute memory.
- alloc
- Stream-ordered allocation helpers.
- blas
- cuBLASLt safe wrapper.
- cudarc_
compat cudarc-shaped compatibility surface — drop-in replacement forcudarc::driver.- cudnn
- cuDNN handle + per-device cache.
- cupti
- CUPTI — safe entry points for the Activity API.
- cusolver
- cuSOLVER Dense handle + per-device cache.
- cusparse
- cuSPARSE handle + per-device cache.
- cutensor
- cuTENSOR handle + per-device cache.
- drv
- Safe CUDA driver layer built directly on
iron_cuda_sys. - events
- Lightweight CUDA timer primitive.
- fft
- cuFFT safe wrapper.
- graph
- CUDA Graphs — thin adapter over
crate::drv::CapturedGraphandcrate::drv::GraphExec. Captures a sequence of kernel launches on a stream, then replays them with a singlecuGraphLaunch. - kernel
- Process-wide kernel cache.
- launch
- IronAccelerator launch helpers over
crate::drv::Function::launch. - nccl
- NCCL safe wrapper.
- observe
- Observability primitives: NVTX scoped ranges for Nsight Systems / Nsight Compute, plus atomic per-session counters that are free to read in the hot path.
- peer
- Peer access management between CUDA devices — NVLink / PCIe.
- pinned
- Pinned (page-locked) host memory pool, built on
crate::drv::PinnedBuf. - pool
- Per-stream small-buffer pool — opt-in fast allocator that recycles freed
DeviceBufs instead of round-tripping throughcuMemAllocAsync/cuMemFreeAsyncon every cycle. - profile
- Built-in profiler — zero-overhead when disabled, GPU-accurate when on.
- rng
- cuRAND safe wrapper.
- safe
- IronAccelerator-curated re-exports of the safe driver surface + helpers.
- streams
- Stream pool — round-robin dispatch across a fixed set of CUDA streams, optionally with priority. Useful when a session needs to overlap copies, compute, and collectives across several engines on the same device.