Struct GpuDevice 

pub struct GpuDevice { /* private fields */ }

Handle to a single CUDA GPU device.

Holds a CUDA context, default stream, and a cached cuBLAS handle. The cuBLAS handle is created once and reused for all matmul/bmm ops, eliminating the ~1.7ms cuModuleLoadData overhead that occurs when creating a new CudaBlas per operation.

Implementations

impl GpuDevice

pub fn new(ordinal: usize) -> GpuResult<Self>

pub fn fork_for_capture(parent: &GpuDevice) -> GpuResult<Self>

Create a GpuDevice with a non-blocking stream forked from the given device’s default stream. The forked stream supports CUDA graph capture (which the legacy default stream does not).

pub fn context(&self) -> &Arc<CudaContext>

pub fn default_stream(&self) -> &Arc<CudaStream>

The device’s default (legacy) stream.

Prefer current_stream which respects the thread-local stream override set by [StreamGuard].

pub fn stream(&self) -> Arc<CudaStream>

The active stream for this device on the current thread.

Returns the thread-local stream set by [StreamGuard] if one is active, otherwise falls back to the device’s default stream. All kernel launches and memory operations should use this.

pub fn blas(&self) -> &CudaBlas

The cached cuBLAS handle — reused for all matmul/bmm operations.

pub fn ordinal(&self) -> usize

impl GpuDevice

pub fn memory_info(&self) -> GpuResult<(usize, usize)>

Query free and total GPU memory for this device.

Returns (free_bytes, total_bytes).

Errors

Returns GpuError::Driver if the CUDA driver call fails.

Trait Implementations

impl Clone for GpuDevice

Available on crate feature cuda only.

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GpuDevice

Available on crate feature cuda only.

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.