Crate ferrox_cuda 

ferrox-cuda: hardware capability detection (always compiled, always tested) plus an optional, feature-gated CUDA execution path.

Build without any GPU support (the default): cargo build -p ferrox-cuda. Build with the CUDA scaffolding included: cargo build -p ferrox-cuda --features cuda.

The cuda feature compiles cleanly in this development sandbox (which has neither a CUDA toolkit nor a GPU) because cudarc is configured for dynamic loading – the driver and NVRTC libraries are dlopen’d at runtime, not linked at build time. That means “this crate compiles with --features cuda” is a true, checked fact. It does not mean the CUDA kernels in gpu.rs have ever executed successfully; see that module’s docs for exactly what has and has not been verified.

Re-exports

pub use capability::HardwareProfile;

pub use capability::SimdCaps;

Modules

capability

Runtime hardware capability detection: probe once, report a plain struct, and let every performance-relevant decision (thread pool width, SIMD kernel selection, GPU residency) derive from the detected machine rather than being hardcoded. Ferrox today only has a CPU execution path, so HardwareProfile::detect() is honest about that: the CUDA fields are always populated (zero / false / None) unless built with --features cuda, and even then they report exactly what ferrox-cuda’s device probe finds, no more.

coalesced_twin

Scalar twin of the coalesced Q4_K matvec kernel.

matvec_kinds

The per-kind CUDA matvec (decode) kernel table. Always compiled, for the same reason mul_mm is: it is kernel text plus three strings per row, and ferrox-core derives its CUDA decode capability from it on builds that do not link cudarc. The per-quant-kind matvec table: which formats CUDA can DECODE, and the three strings a launch needs for each.

mul_mm

The mul_mm kernel source and its per-quant-kind dispatch table. Always compiled: it is CUDA C text plus a scalar twin, neither of which needs cudarc, so the default cargo test -p ferrox-cuda run on a GPU-less host still exercises the arithmetic the kernel encodes. Only the launch path ([mul_mm_launch]) is feature-gated. mul_mm: a batched quantized GEMM for CUDA – the CUDA C source and the per-quant-kind dispatch table.

mul_mm_kinds

One module per quant-format family, holding the mul_mm::KINDS rows. Split out of mul_mm.rs so that adding a format is a new file rather than a new section of a file nobody wants to open. The per-quant-kind rows of the CUDA mul_mm dispatch table, one module per format family.

mul_mm_ref

The scalar twin of the CUDA mul_mm kernel: the same GEMM, executed on the host, one emulated threadblock at a time.