Skip to main content

Crate ironaccelerator_cuda

Crate ironaccelerator_cuda 

Source
Expand description

§ironaccelerator-cuda

Low-level, hardware-agnostic CUDA interface. Targets the CUDA Toolkit 13.2 API surface via iron_cuda_sys, with safe wrappers around the driver, NVRTC, and every major vendor library (cuBLAS / cuBLASLt / cuDNN / cuRAND / cuSPARSE / cuSOLVER / cuFFT / cuTENSOR / NCCL / NVTX / CUPTI).

Scope. This crate is a driver substrate. It does not ship kernels, workload planners, FP8 recipes, attention/MoE implementations, or autotuners — those belong to downstream libraries layered on top. What’s here:

§If you want to…

TaskUse
port existing cudarc 0.19 codecudarc_compat (drop-in use swap)
open a device + create a streamdrv::Device::open then drv::Stream::new
allocate a typed device bufferdrv::DeviceBuf::alloc / alloc::alloc
copy host ↔ device on a streamdrv::DeviceBuf::copy_from_host / copy_to_host
compile a CUDA C++ kernel at runtimekernel::get_or_compile (cached on disk per arch+source)
launch a kernel with typed argsdrv::Function::launch or launch::launch_1d / _2d
record + measure GPU timeevents::Timer (or drv::TimingEvent directly)
capture & replay a stream as a CUDA graphdrv::Stream::begin_capture → drv::GraphExec
call cuBLASLt / cuDNN / cuFFT / NCCLthe matching blas/cudnn/fft/nccl module
drop down to raw FFIsys (re-exports iron_cuda_sys)

§Performance posture

Every hot path is #[inline]. The driver fns() lookup uses an AtomicPtr<DriverFns> fast-path cache, and each Device/Stream/Event/ Module/Function caches &'static DriverFns at construction — so a wrapped op costs one struct-field load + one FFI entry. See the workspace README for benchmarks against cudarc 0.19; alloc/free is ~2× faster.

Re-exports§

pub use iron_cuda_sys as sys;

Modules§

advanced
CUDA 12.3+/12.4+/13.x advanced primitives: virtual memory, multicast, green contexts, graph conditional nodes, confidential-compute memory.
alloc
Stream-ordered allocation helpers.
blas
cuBLASLt safe wrapper.
cudarc_compat
cudarc-shaped compatibility surface — drop-in replacement for cudarc::driver.
cudnn
cuDNN handle + per-device cache.
cupti
CUPTI — safe entry points for the Activity API.
cusolver
cuSOLVER Dense handle + per-device cache.
cusparse
cuSPARSE handle + per-device cache.
cutensor
cuTENSOR handle + per-device cache.
drv
Safe CUDA driver layer built directly on iron_cuda_sys.
events
Lightweight CUDA timer primitive.
fft
cuFFT safe wrapper.
graph
CUDA Graphs — thin adapter over crate::drv::CapturedGraph and crate::drv::GraphExec. Captures a sequence of kernel launches on a stream, then replays them with a single cuGraphLaunch.
kernel
Process-wide kernel cache.
launch
IronAccelerator launch helpers over crate::drv::Function::launch.
nccl
NCCL safe wrapper.
observe
Observability primitives: NVTX scoped ranges for Nsight Systems / Nsight Compute, plus atomic per-session counters that are free to read in the hot path.
peer
Peer access management between CUDA devices — NVLink / PCIe.
pinned
Pinned (page-locked) host memory pool, built on crate::drv::PinnedBuf.
pool
Per-stream small-buffer pool — opt-in fast allocator that recycles freed DeviceBufs instead of round-tripping through cuMemAllocAsync / cuMemFreeAsync on every cycle.
profile
Built-in profiler — zero-overhead when disabled, GPU-accurate when on.
rng
cuRAND safe wrapper.
safe
IronAccelerator-curated re-exports of the safe driver surface + helpers.
streams
Stream pool — round-robin dispatch across a fixed set of CUDA streams, optionally with priority. Useful when a session needs to overlap copies, compute, and collectives across several engines on the same device.