Trait Backend 

pub trait Backend: Send + Sync {
    type Weights: Send + Sync;
    type Plan: Send;

    // Required methods
    fn caps(&self) -> Caps;
    fn upload(
        &self,
        tensors: &[HostTensor<'_>],
        graph: &Graph,
    ) -> Result<Self::Weights>;
    fn lower(
        &self,
        w: &Self::Weights,
        graph: &Graph,
        bucket: Bucket,
    ) -> Result<Self::Plan>;
    fn run(
        &self,
        plan: &mut Self::Plan,
        batch: &Batch<'_>,
        out: &mut Outputs,
    ) -> Result<()>;

    // Provided methods
    fn weight_bytes(&self, _w: &Self::Weights) -> usize { ... }
    fn plan_bytes(&self, _p: &Self::Plan) -> usize { ... }
}

A device that runs graphs. The engine is generic over it, so the path through it is monomorphized and there is no dynamic dispatch per op.

Required Associated Types

type Weights: Send + Sync

Weights in the backend’s own layout.

type Plan: Send

A graph lowered for one bucket, with its arena.

Required Methods

fn caps(&self) -> Caps

What the backend can do.

fn upload( &self, tensors: &[HostTensor<'_>], graph: &Graph, ) -> Result<Self::Weights>

Converts a checkpoint’s tensors, indexed as the graph’s weights are.

Errors

When a tensor cannot be converted.

fn lower( &self, w: &Self::Weights, graph: &Graph, bucket: Bucket, ) -> Result<Self::Plan>

Lowers a graph for one bucket. This is where memory is allocated.

Errors

Error::Unsupported for an op or a shape the backend cannot run.

fn run( &self, plan: &mut Self::Plan, batch: &Batch<'_>, out: &mut Outputs, ) -> Result<()>

Runs a batch that fits the plan’s bucket and has been checked.

Errors

When the device fails.

Provided Methods

fn weight_bytes(&self, _w: &Self::Weights) -> usize

Bytes the weights take on the device.

fn plan_bytes(&self, _p: &Self::Plan) -> usize

Bytes a plan takes on the device, its arena and staging buffers.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors