Struct Buffer 

pub struct Buffer { /* private fields */ }

Data storring buffer.

Deletes data on drop.

Implementations

impl Buffer

Buffer creation section.

pub fn try_clone(&self) -> Result<Self, Error>

Try clone buffer content.

Note: If memory type is not Cpu, should be called in sync with cuda context pinned (check module level documentation for more info).

pub fn alloc<T: Sample>( count: usize, memory_type: MemoryType, ) -> Result<Self, Error>

Allocate new buffer of requested memory type.
count: size of buffer in T units (i.e. 128 chunks of f32 (that has byte size 512) should be allocated with count=128).
memory_type: Cpu/Pinned/Gpu.

Note: If memory type is not Cpu, should be called in sync with cuda context pinned (check module level documentation for more info).

pub fn alloc_with_data_type( count: usize, memory_type: MemoryType, data_type: DataType, ) -> Result<Self, Error>

Allocate new buffer of requested memory type.
count: size of buffer in T units (i.e. 128 chunks of f32 (that has byte size 512) should be allocated with count=128).
memory_type: Cpu/Pinned/Gpu.
data_type: type of Samples.

Note: If memory type is not Cpu, should be called in sync with cuda context pinned (check module level documentation for more info).

pub fn from<T: Sample, S: AsRef<[T]>>(slice: S) -> Self

Create CPU buffer of data type T::DARA_TYPE from slice of T.

impl Buffer

Buffer metadata section.

pub fn memory_type(&self) -> MemoryType

Get memory type of storred data.

pub fn data_type(&self) -> DataType

Get data type of storred data.

pub fn size(&self) -> usize

Get byte size of data.

pub fn is_empty(&self) -> bool

True if not containing any data.

impl Buffer

Buffer data permutation section.

pub fn copy_from_slice<S: AsRef<[T]>, T: Sample>( &mut self, offset: usize, source: S, ) -> Result<(), Error>

Copy source content to self from the offset position.
Returns error if offset + size_of_val(source) > self.size().

offset: offset (in bytes) from the beginning of the Buffer to location to copy source to. source: slice of Samples.

Note: If self.memory_type is not Cpu, should be called in sync with cuda context pinned (check module level documentation for more info).

pub fn copy_from_cuda_array( &mut self, offset: usize, source: CudaArray, ) -> Result<(), Error>

Copy source content to self from the offset position.
Returns error if offset + source.len > self.size().

offset: offset (in bytes) from the beginning of the Buffer to location to copy source to. source: cuda array.

Note: This method should be called in sync with cuda context pinned (check module level documentation for more info).

pub fn into_cpu(self) -> Result<Self, Error>

Move this Buffer content to CPU memory.

Note: If self.memory_type() is not Cpu, method should be called in sync with cuda context pinned (check module level documentation for more info).

pub fn into_pinned(self) -> Result<Self, Error>

Move this Buffer content to Pinned memory.

Note: This method should be called in sync with cuda context pinned (check module level documentation for more info).

pub fn into_gpu(self) -> Result<Self, Error>

Move this Buffer content to Gpu memory.

Note: This method should be called in sync with cuda context pinned (check module level documentation for more info).

impl Buffer

Obtaining buffer content section.

pub fn bytes(&self) -> &[u8]

Get buffer content as bytes.

Will return nothing if self.memory_type == Gpu. Use Buffer::get_owned_slice instead.

pub fn bytes_mut(&mut self) -> &mut [u8]

Get buffer mutable content as bytes.

Will return nothing if self.memory_type == Gpu. Buffer::get_cuda_array can be used instead to implement this logic.

pub fn get_owned_slice<Range: RangeBounds<usize> + Debug>( &self, range: Range, ) -> Result<Vec<u8>, Error>

Get content of the buffer as host located bytes.
range: part of the buffer to return.

pub unsafe fn get_cuda_array(&self) -> CudaArray

Get content of the GPU based buffer.

Panics

Panics if self.memory_type != Gpu.

Safety

Returned struct points to the same location as buffer, so the rules are the same as sharing *mut on an object.
Be careful: Buffer will delete data on drop, so be afraid of double memory free. Also any shinenigans with the data during the inference are forbidden: Triton must have exclusive write ascess to data during the inference.

Trait Implementations

impl<T: Sample> AsMut<[T]> for Buffer

fn as_mut(&mut self) -> &mut [T]

Converts this type into a mutable reference on the slice of T.

Will return nothing if self.memory_type == Gpu.

Panics

Panics if T does not match Buffer data type.

impl<T: Sample> AsRef<[T]> for Buffer

fn as_ref(&self) -> &[T]

Converts this type into a shared reference on the slice of T.

Will return nothing if self.memory_type == Gpu.

Panics

Panics if T does not match Buffer data type.

impl Debug for Buffer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for Buffer

fn drop(&mut self)

Executes the destructor for this type.

impl From<Buffer> for CudaArray

Available on crate feature gpu only.

Create CudaArray from Buffer.

Buffer destructor will not be invoked so data will be safe.

fn from(value: Buffer) -> CudaArray

Converts to this type from the input type.

impl From<CudaArray> for Buffer

Available on crate feature gpu only.

Create GPU buffers of DataType::Uint8 from CudaArray. Result memory type will be MemoryType::Gpu.

Note that nothing is allocated on this call, meaning that result buffer will just point on data provided by argument.

fn from(value: CudaArray) -> Self

Converts to this type from the input type.

impl Send for Buffer

impl Sync for Buffer