Trait ComputeServer

pub trait ComputeServer:
    Send
    + Debug
    + Sized {
    type Kernel: KernelMetadata;
    type Info: Debug + Send + Sync;
    type Storage: ComputeStorage;
    type Feature: Ord + Copy + Debug + Send + Sync;

    // Required methods
    fn read(
        &mut self,
        bindings: Vec<Binding>,
    ) -> Pin<Box<dyn Future<Output = Vec<Vec<u8>>> + Send>>;
    fn read_tensor(
        &mut self,
        bindings: Vec<BindingWithMeta>,
    ) -> Pin<Box<dyn Future<Output = Vec<Vec<u8>>> + Send>>;
    fn sync(&mut self) -> Pin<Box<dyn Future<Output = ()> + Send>>;
    fn get_resource(
        &mut self,
        binding: Binding,
    ) -> BindingResource<<Self::Storage as ComputeStorage>::Resource>;
    fn create(&mut self, data: &[u8]) -> Handle;
    fn create_tensors(
        &mut self,
        data: Vec<&[u8]>,
        shapes: Vec<&[usize]>,
        elem_sizes: Vec<usize>,
    ) -> Vec<(Handle, Vec<usize>)>;
    fn empty(&mut self, size: usize) -> Handle;
    fn empty_tensors(
        &mut self,
        shapes: Vec<&[usize]>,
        elem_sizes: Vec<usize>,
    ) -> Vec<(Handle, Vec<usize>)>;
    unsafe fn execute(
        &mut self,
        kernel: Self::Kernel,
        count: CubeCount,
        bindings: Bindings,
        kind: ExecutionMode,
        logger: Arc<ServerLogger>,
    );
    fn flush(&mut self);
    fn memory_usage(&self) -> MemoryUsage;
    fn memory_cleanup(&mut self);
    fn start_profile(&mut self) -> ProfilingToken;
    fn end_profile(
        &mut self,
        token: ProfilingToken,
    ) -> Result<ProfileDuration, ProfileError>;
}

The compute server is responsible for handling resources and computations over resources.

Everything in the server is mutable, therefore it should be solely accessed through the compute channel for thread safety.

Required Associated Types

type Kernel: KernelMetadata

The kernel type defines the computation algorithms.

type Info: Debug + Send + Sync

Information that can be retrieved for the runtime.

type Storage: ComputeStorage

The storage type defines how data is stored and accessed.

type Feature: Ord + Copy + Debug + Send + Sync

The type of the features supported by the server.

Required Methods

fn read( &mut self, bindings: Vec<Binding>, ) -> Pin<Box<dyn Future<Output = Vec<Vec<u8>>> + Send>>

Given bindings, returns the owned resources as bytes.

fn read_tensor( &mut self, bindings: Vec<BindingWithMeta>, ) -> Pin<Box<dyn Future<Output = Vec<Vec<u8>>> + Send>>

Given tensor handles, returns the owned resources as bytes.

fn sync(&mut self) -> Pin<Box<dyn Future<Output = ()> + Send>>

Wait for the completion of every task in the server.

fn get_resource( &mut self, binding: Binding, ) -> BindingResource<<Self::Storage as ComputeStorage>::Resource>

Given a resource handle, returns the storage resource.

fn create(&mut self, data: &[u8]) -> Handle

Given a resource as bytes, stores it and returns the memory handle.

fn create_tensors( &mut self, data: Vec<&[u8]>, shapes: Vec<&[usize]>, elem_sizes: Vec<usize>, ) -> Vec<(Handle, Vec<usize>)>

Given a resource as bytes with shape, stores it and returns the tensor handle. May or may not be contiguous, depending on what’s best for the given runtime. Always use strides to index. For example, in CUDA, this will allocate a padded tensor where the last dimension is padded to the cache lines, so row access is faster.

fn empty(&mut self, size: usize) -> Handle

Reserves size bytes in the storage, and returns a handle over them.

fn empty_tensors( &mut self, shapes: Vec<&[usize]>, elem_sizes: Vec<usize>, ) -> Vec<(Handle, Vec<usize>)>

Reserves shape bytes in the storage, and returns a handle to it.

unsafe fn execute( &mut self, kernel: Self::Kernel, count: CubeCount, bindings: Bindings, kind: ExecutionMode, logger: Arc<ServerLogger>, )

Executes the kernel over the given memory handles.

Kernels have mutable access to every resource they are given and are responsible of determining which should be read or written.

Safety

When executing with mode ExecutionMode::Unchecked, out-of-bound reads and writes can happen.

fn flush(&mut self)

Flush all outstanding tasks in the server.

fn memory_usage(&self) -> MemoryUsage

The current memory usage of the server.

fn memory_cleanup(&mut self)

Ask the server to release memory that it can release.

fn start_profile(&mut self) -> ProfilingToken

Enable collecting timestamps.

fn end_profile( &mut self, token: ProfilingToken, ) -> Result<ProfileDuration, ProfileError>

Disable collecting timestamps.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors