Struct CudaBackend 

pub struct CudaBackend { /* private fields */ }

One GPU, with its stream, kernels and cuBLASLt handle.

Implementations

impl CudaBackend

pub fn new(ordinal: usize, precision: Precision) -> Result<Self>

Opens GPU ordinal and compiles the kernels for it.

Errors

Error::Device when there is no such GPU, no driver, or the kernels do not compile.

pub fn name(&self) -> &str

The GPU’s name, as the driver reports it.

pub fn precision(&self) -> Precision

What runs in FP16.

pub fn arch(&self) -> (i32, i32)

Compute capability, major and minor.

Trait Implementations

impl Backend for CudaBackend

type Weights = Weights

Weights in the backend’s own layout.

type Plan = CudaPlan

A graph lowered for one bucket, with its arena.

fn caps(&self) -> Caps

What the backend can do.

fn weight_bytes(&self, w: &Weights) -> usize

Bytes the weights take on the device.

fn plan_bytes(&self, p: &CudaPlan) -> usize

Bytes a plan takes on the device, its arena and staging buffers.

fn upload(&self, tensors: &[HostTensor<'_>], _graph: &Graph) -> Result<Weights>

Converts a checkpoint’s tensors, indexed as the graph’s weights are.

fn lower(&self, w: &Weights, graph: &Graph, bucket: Bucket) -> Result<CudaPlan>

Lowers a graph for one bucket. This is where memory is allocated.

fn run( &self, p: &mut CudaPlan, batch: &Batch<'_>, out: &mut Outputs, ) -> Result<()>

Runs a batch that fits the plan’s bucket and has been checked.

impl Debug for CudaBackend

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.