Struct GpuRuntime 

pub struct GpuRuntime {
    pub device: GpuDeviceInfo,
    pub devices: Vec<GpuDeviceInfo>,
    pub policy: GpuDispatchPolicy,
    pub memory_budget_bytes: usize,
}

Fields

device: GpuDeviceInfo

Highest-scoring probed CUDA device. Existing dispatch code routes one-shot kernels through this device.

devices: Vec<GpuDeviceInfo>

All usable CUDA devices discovered at probe time, ordered by score.

policy: GpuDispatchPolicy

memory_budget_bytes: usize

Implementations

impl GpuRuntime

pub fn probe() -> Result<GpuAvailability, GpuError>

pub fn availability() -> Result<GpuAvailabilityRef<'static>, GpuError>

Return the cached probe outcome without collapsing faults into absence.

pub fn resolve(policy: GpuPolicy) -> Result<Option<&'static Self>, GpuError>

Resolve CUDA under an explicit policy. Ok(None) is reserved for a genuine absence under Auto/Off; probe faults always remain Err, and Required converts absence into RequiredDeviceUnavailable.

pub fn require() -> Result<&'static Self, GpuError>

Resolve CUDA under Required semantics and return the device handle.

pub fn resolution_call_count() -> u64

Number of times Self::availability has been entered process-wide.

Test-facing instrumentation for the laziness contract: a size-gated caller that returns before resolving availability leaves this unchanged, so a test can assert a CPU-sized decision path created no CUDA context. This is a monotone call counter, NOT a probe-success flag.

pub fn resolve_if_dense_work_exceeds_floor( policy: GpuPolicy, work_flops: u128, ) -> Result<Option<&'static Self>, GpuError>

Size-gated Self::resolve: resolve the process-wide runtime only when the estimated dense arithmetic work_flops clears the GPU-dispatch flop floor.

This is the ordering fix for the CUDA startup tax. For a CPU-sized problem (work_flops below the floor) it returns Ok(None) without calling Self::resolve, so the device probe — and the cuDevicePrimaryCtxRetain primary-context creation it performs on every GPU — never runs. The problem-size decision therefore strictly precedes any driver contact, and a CPU-sized fit pays ZERO CUDA cost.

The floor is GpuDispatchPolicy::MIN_CALIBRATABLE_GEMM_FLOPS — the smallest gemm_min_flops ANY reachable policy (default seed or device-calibrated) can carry, known WITHOUT a device — so the gate never needs a probe to decide it should not probe, and refusing below it can never block work that any policy would have dispatched. Work at or above the floor falls through to the identical lossless resolution path (where the real, possibly calibrated policy still gates each op), so device behaviour for genuinely GPU-sized problems is unchanged.

pub fn resolve_if_fused_batch_exceeds_floor( policy: GpuPolicy, rows: usize, ) -> Result<Option<&'static Self>, GpuError>

Size-gated Self::resolve for independent fused row kernels.

Batches below GpuDispatchPolicy::MIN_CALIBRATABLE_FUSED_KERNEL_N cannot be admitted by either the default or any device-calibrated policy. Refuse them before availability resolution so a CPU-sized first call does not create CUDA contexts and run calibration merely to learn that it should stay on the CPU. At and above the universal floor, the concrete runtime’s calibrated policy remains authoritative.

pub fn policy(&self) -> &GpuDispatchPolicy

pub fn selected_device(&self) -> &GpuDeviceInfo

impl GpuRuntime

pub fn device_ordinals(&self) -> Vec<usize>

Ordinals of all usable devices, highest-score first.

self.devices is already score-sorted at probe time, so this simply projects out the ordinals. Empty only if the probe somehow produced no devices (the public probe() guarantees at least one on Ok(Some(_))).

pub fn device_count(&self) -> usize

Number of usable devices in the pool.

pub fn memory_budget_for(&self, ordinal: usize) -> usize

Per-device byte budget: free memory capped at half of total, matching the primary-device budget computed in device_runtime::probe. Falls back to the primary memory_budget_bytes when the ordinal is not in the pool so a caller that passes a stale ordinal still gets a usable (conservative) budget rather than zero.

Trait Implementations

impl Clone for GpuRuntime

fn clone(&self) -> GpuRuntime

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GpuRuntime

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.