Struct Cuda 

pub struct Cuda;

The CUDA backend (ferrox-cuda).

Trait Implementations

impl BackendCaps for Cuda

const GEMM_FALLBACK: &'static str = "CUDA per-position matvec"

apply_batch_with_acts decomposes a CUDA prefill into one matvec per position for every kind off Cuda::gemm_supported.

fn matvec_kernel(kind: QuantKind) -> Option<&'static str>

The decode path, and the arm that has actually run on a GPU.

Wider than Cuda::gemm_supported. The name is returned only to share BackendCaps::matvec_kernel’s shape with Metal; nothing on the CUDA path reads it, because ferrox-cuda’s launchers are named functions rather than entries in a string-keyed table.

fn gemm_supported(kind: QuantKind) -> bool

The mul_mm prefill path.

Deliberately narrower than Cuda::matvec_kernel: ferrox-cuda had no matrix-matrix product at all until Q8_0 and Q4_0 landed, so every other kind still decomposes a prefill into per-position matvecs.

Stated here rather than delegating to ferrox_cuda::mul_mm::kind_by_name, because ferrox-cuda is only a dependency under the cuda feature and this predicate is compiled unconditionally (the capability report reads it on every build).

Two tables that must agree about one set is the failure this codebase keeps paying for, so the agreement is a TEST rather than a hope: the_cuda_gemm_kinds_match_the_kernel_table runs under --features cuda and compares this against kind_by_name for every QuantKind.

UNRUN ON HARDWARE. The kernel is checked against a scalar twin and by executing the emitted CUDA C on the host, and has never executed on a GPU. See crates/ferrox-cuda/src/mul_mm.rs.

const ID: Backend = Backend::Cuda

How crate::kernel_registry reports this backend. Dispatch and observability read the same constant, so a backend cannot be dispatched to under one name and reported under another.

const NAME: &'static str = "CUDA"

Human-readable name, for the one message a dispatch failure prints.