Struct ComputeClient

pub struct ComputeClient<Server, Channel>
where
    Server: ComputeServer,
{ /* private fields */ }

The ComputeClient is the entry point to require tasks from the ComputeServer. It should be obtained for a specific device via the Compute struct.

Implementations

impl<Server, Channel> ComputeClient<Server, Channel>

where Server: ComputeServer, Channel: ComputeChannel<Server>,

pub fn info(&self) -> &<Server as ComputeServer>::Info

Get the info of the current backend.

pub fn new( channel: Channel, properties: DeviceProperties<<Server as ComputeServer>::Feature>, info: <Server as ComputeServer>::Info, ) -> ComputeClient<Server, Channel>

Create a new client.

pub async fn read_async(&self, bindings: Vec<Binding>) -> Vec<Vec<u8>>

Given bindings, returns owned resources as bytes.

pub fn read(&self, bindings: Vec<Binding>) -> Vec<Vec<u8>>

Given bindings, returns owned resources as bytes.

Remarks

Panics if the read operation fails.

pub async fn read_one_async(&self, binding: Binding) -> Vec<u8>

Given a binding, returns owned resource as bytes.

pub fn read_one(&self, binding: Binding) -> Vec<u8>

Given a binding, returns owned resource as bytes.

Remarks

Panics if the read operation fails.

pub async fn read_tensor_async( &self, bindings: Vec<BindingWithMeta>, ) -> Vec<Vec<u8>>

Given bindings, returns owned resources as bytes.

pub fn read_tensor(&self, bindings: Vec<BindingWithMeta>) -> Vec<Vec<u8>>

Given bindings, returns owned resources as bytes.

Remarks

Panics if the read operation fails.

The tensor must be in the same layout as created by the runtime, or more strict. Contiguous tensors are always fine, strided tensors are only ok if the stride is similar to the one created by the runtime (i.e. padded on only the last dimension). A way to check stride compatiblity on the runtime will be added in the future.

Also see ComputeClient::create_tensor.

pub async fn read_one_tensor_async(&self, binding: BindingWithMeta) -> Vec<u8>

Given a binding, returns owned resource as bytes. See ComputeClient::read_tensor

pub fn read_one_tensor(&self, binding: BindingWithMeta) -> Vec<u8>

Given a binding, returns owned resource as bytes.

Remarks

Panics if the read operation fails. See ComputeClient::read_tensor

pub fn get_resource( &self, binding: Binding, ) -> BindingResource<<<Server as ComputeServer>::Storage as ComputeStorage>::Resource>

Given a resource handle, returns the storage resource.

pub fn create(&self, data: &[u8]) -> Handle

Given a resource, stores it and returns the resource handle.

pub fn create_tensor( &self, data: &[u8], shape: &[usize], elem_size: usize, ) -> (Handle, Vec<usize>)

Given a resource and shape, stores it and returns the tensor handle and strides. This may or may not return contiguous strides. The layout is up to the runtime, and care should be taken when indexing.

Currently the tensor may either be contiguous (most runtimes), or “pitched”, to use the CUDA terminology. This means the last (contiguous) dimension is padded to fit a certain alignment, and the strides are adjusted accordingly. This can make memory accesses significantly faster since all rows are aligned to at least 16 bytes (the maximum load width), meaning the GPU can load as much data as possible in a single instruction. It may be aligned even more to also take cache lines into account.

However, the stride must be taken into account when indexing and reading the tensor (also see ComputeClient::read_tensor).

pub fn empty(&self, size: usize) -> Handle

Reserves size bytes in the storage, and returns a handle over them.

pub fn empty_tensor( &self, shape: &[usize], elem_size: usize, ) -> (Handle, Vec<usize>)

Reserves shape in the storage, and returns a tensor handle for it. See ComputeClient::create_tensor

pub fn execute( &self, kernel: <Server as ComputeServer>::Kernel, count: CubeCount, bindings: Bindings, )

Executes the kernel over the given bindings.

pub unsafe fn execute_unchecked( &self, kernel: <Server as ComputeServer>::Kernel, count: CubeCount, bindings: Bindings, )

Executes the kernel over the given bindings without performing any bound checks.

Safety

Without checks, the out-of-bound reads and writes can happen.

pub fn flush(&self)

Flush all outstanding commands.

pub async fn sync(&self)

Wait for the completion of every task in the server.

pub fn properties( &self, ) -> &DeviceProperties<<Server as ComputeServer>::Feature>

Get the features supported by the compute server.

pub fn memory_usage(&self) -> MemoryUsage

Get the current memory usage of this client.

pub fn memory_cleanup(&self)

Ask the client to release memory that it can release.

Nb: Results will vary on what the memory allocator deems beneficial, so it’s not guaranteed any memory is freed.

pub fn profile(&self, func: impl FnOnce()) -> ProfileDuration

Measure the execution time of some inner operations.

Nb: this function will only allow one function at a time to be submitted when multithrading. Recursive measurements are not allowed and will deadlock.

Trait Implementations

impl<S, C> Clone for ComputeClient<S, C>

where S: ComputeServer, C: ComputeChannel<S>,

fn clone(&self) -> ComputeClient<S, C>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<Server, Channel> Debug for ComputeClient<Server, Channel>

where Server: Debug + ComputeServer, Channel: Debug,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.