Struct CubeTensor 

pub struct CubeTensor<R: CubeRuntime> {
    pub client: ComputeClient<R>,
    pub handle: Handle,
    pub shape: Shape,
    pub device: R::Device,
    pub strides: Vec<usize>,
    pub dtype: DType,
    pub qparams: Option<QParams>,
}

The basic tensor primitive struct.

Fields

client: ComputeClient<R>

Compute client for the runtime.

handle: Handle

The buffer where the data are stored.

shape: Shape

The shape of the tensor.

device: R::Device

The device of the tensor.

strides: Vec<usize>

The strides of the tensor.

dtype: DType

The datatype of the tensor.

qparams: Option<QParams>

Runtime quantization parameters, if applicable

Implementations

impl<R> CubeTensor<R>

where R: CubeRuntime,

pub fn new( client: ComputeClient<R>, handle: Handle, shape: Shape, device: R::Device, strides: Vec<usize>, dtype: DType, ) -> Self

Create a new standard tensor

pub fn new_contiguous( client: ComputeClient<R>, device: R::Device, shape: Shape, handle: Handle, dtype: DType, ) -> Self

Create a new tensor with a contiguous memory layout.

pub fn to_client(&self, client: ComputeClient<R>, device: R::Device) -> Self

Change the context of the current tensor and return the newly transferred tensor.

pub fn as_handle_ref(&self) -> TensorHandleRef<'_, R>

Return the reference to a tensor handle.

pub fn elem_size(&self) -> usize

Returns the element size of this tensor

pub fn as_tensor_arg<'a>(&'a self, line_size: LineSize) -> TensorArg<'a, R>

Return the reference to a tensor argument.

pub fn as_array_arg<E: CubeElement>( &self, line_size: LineSize, ) -> ArrayArg<'_, R>

Return the reference to an array argument.

pub fn try_scheme(&self) -> Option<&QuantScheme>

Return the QuantScheme if present

pub fn copy(&self) -> Self

Copy the current tensor.

pub fn can_mut(&self) -> bool

Check if the tensor is safe to mutate.

pub fn assert_is_on_same_device(&self, other: &Self)

Assert that both tensors are on the same device.

pub fn is_contiguous(&self) -> bool

Check if the current tensor is contiguous.

A tensor is contiguous if the elements are stored in memory if the strides in non-increasing order and the strides at position k is equal to the product of the shapes at all positions greater than k. However, all axes with a shape of 1 are ignored.

pub fn is_contiguous_buffer(&self) -> bool

Check if the current tensor has a contiguous backing buffer (no overlap and no empty memory regions within the shape).

impl<R: CubeRuntime> CubeTensor<R>

pub fn new_quantized( client: ComputeClient<R>, handle: Handle, shape: Shape, device: R::Device, strides: Vec<usize>, dtype: DType, qparams: QParams, ) -> Self

Create a new quantized tensor

pub fn quantized_handles(&self) -> Option<(CubeTensor<R>, CubeTensor<R>)>

Returns the two tensors: (values, params) for a quantized tensor. For the values, native types that aren’t supported as a normal DType will be returned as an unsigned integer tensor representing the bits. Should be reconstructed using from_bits in kernels.

pub fn scales(&self) -> Option<CubeTensor<R>>

Construct a separate tensor for the quantization scales, if present

Trait Implementations

impl<R> Clone for CubeTensor<R>

where R: CubeRuntime,

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<R> Debug for CubeTensor<R>

where R: CubeRuntime,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<R: CubeRuntime> From<CubeTensor<R>> for CubeFusionHandle<R>

Available on crate features fusion only.

fn from(value: CubeTensor<R>) -> Self

Converts to this type from the input type.

impl<R: CubeRuntime> From<CubeTensor<R>> for TensorHandle<R>

fn from(val: CubeTensor<R>) -> Self

Converts to this type from the input type.

impl<R: CubeRuntime> QTensorPrimitive for CubeTensor<R>

fn scheme(&self) -> &QuantScheme

Returns the quantization settings for the given tensor.

fn acc_precision(&self) -> QuantAcc

The precision used for the accumulation in various kernels.

fn propagation(&self) -> QuantPropagation

How quantization is propagated during computation.

fn default_scheme() -> QuantScheme

Returns the default tensor quantization scheme.

impl<R: CubeRuntime> TensorMetadata for CubeTensor<R>

fn dtype(&self) -> DType

The dtype of the tensor.

fn shape(&self) -> Shape

The shape of the tensor.

fn rank(&self) -> usize

The number of dimensions of the tensor.

impl<R: CubeRuntime> AutotuneOutput for CubeTensor<R>