Struct CubeTensor 

pub struct CubeTensor {
    pub client: Client,
    pub handle: Handle,
    pub meta: Box<Metadata>,
    pub device: CubeDevice,
    pub dtype: DType,
    pub qparams: Option<QParams>,
}

The basic tensor primitive struct.

Fields

client: Client

Compute client for the runtime this tensor’s device names.

handle: Handle

The buffer where the data are stored.

meta: Box<Metadata>

The metadata of the tensor.

device: CubeDevice

The device of the tensor.

dtype: DType

The datatype of the tensor.

qparams: Option<QParams>

Runtime quantization parameters, if applicable

Implementations

impl CubeTensor

pub fn new( client: Client, handle: Handle, metadata: Metadata, device: CubeDevice, dtype: DType, ) -> Self

Create a new standard tensor

pub fn new_contiguous( client: Client, device: CubeDevice, shape: Shape, handle: Handle, dtype: DType, ) -> Self

Create a new tensor with a contiguous memory layout.

pub fn to_client(&mut self, client: Client, device: CubeDevice) -> Self

Change the context of the current tensor and return the newly transferred tensor.

pub fn binding(self) -> TensorBinding

Return the reference to a tensor handle.

pub fn elem_size(&self) -> usize

Returns the element size of this tensor

pub fn into_tensor_arg(self) -> TensorArg

Return the reference to a tensor argument.

Panics

On a storage-tiled tensor: a kernel argument is read as rows, and only cubek’s matmul reads storage tiles, through binding. Un-tile it first (untile).

pub fn into_buffer_arg(self) -> BufferArg

Return the reference to a buffer argument.

pub fn as_tensor_alias(&self, input_pos: usize) -> TensorArg

Returns a reference to the aliased tensor argument.

pub fn into_linear_view(self) -> LinearViewLaunch

Return a linear view of this tensor.

pub fn as_linear_view_alias(&self, input_pos: usize) -> LinearViewLaunch

Return an aliased linear view of this tensor

pub fn into_linear_view_like(self, reference: &Self) -> LinearViewLaunch

Return a linear view broadcast to the reference tensor’s shape

pub fn required_address_type(&self) -> AddressType

Returns the address type required to index this tensor

pub fn try_scheme(&self) -> Option<&QuantScheme>

Return the QuantScheme if present

pub fn copy(&self) -> Self

Copy the current tensor.

pub fn can_mut(&self) -> bool

Check if the tensor is safe to mutate.

pub fn assert_is_on_same_device(&self, other: &Self)

Assert that both tensors are on the same device.

pub fn is_contiguous(&self) -> bool

Check if the current tensor is contiguous.

A tensor is contiguous if the elements are stored in memory if the strides in non-increasing order and the strides at position k is equal to the product of the shapes at all positions greater than k. However, all axes with a shape of 1 are ignored.

pub fn is_contiguous_buffer(&self) -> bool

Check if the current tensor has a contiguous backing buffer (no overlap and no empty memory regions within the shape).

pub fn is_nonoverlapping(&self) -> bool

Checks if the tensor is non-overlapping (can be safely written to).

impl CubeTensor

pub fn new_quantized( client: Client, handle: Handle, shape: Shape, device: CubeDevice, strides: Strides, dtype: DType, qparams: QParams, ) -> Self

Create a new quantized tensor

pub fn quantized_handles(&self) -> Option<(CubeTensor, CubeTensor)>

Returns the two tensors: (values, params) for a quantized tensor. For the values, native types that aren’t supported as a normal DType will be returned as an unsigned integer tensor representing the bits. Should be reconstructed using from_bits in kernels.

A storage-tiled tensor’s values keep its metadata whole: its physical dims already state the packing, and only a kernel that reads tiles may bind them — any other refuses them as rows. The scales keep their own metadata, tiled or not.

pub fn scales(&self) -> Option<CubeTensor>

Construct a separate tensor for the quantization scales, if present

pub fn global(&self) -> Option<CubeTensor>

Construct a separate tensor for the per-tensor scale, for a two-level scheme.

Trait Implementations

impl AutotuneOutput for CubeTensor

impl Clone for CubeTensor

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CubeTensor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<CubeTensor> for TensorHandle

fn from(val: CubeTensor) -> Self

Converts to this type from the input type.

impl From<CubeTensor> for CubeFusionHandle

Available on crate features fusion only.

fn from(value: CubeTensor) -> Self

Converts to this type from the input type.

impl TensorMetadata for CubeTensor

fn shape(&self) -> Shape

The logical shape: a storage-tiled tensor’s physical dims split each matrix dim in two, and its own metadata folds them back. Identity on a plain tensor.

type Device = Device

The device type associated with the tensor.

fn dtype(&self) -> DType

Get the dtype of the tensor.

fn rank(&self) -> usize

Get the number of dimensions of the tensor.

fn device(&self) -> Self::Device

Get the device associated with the tensor.

fn can_mut(&self) -> bool

Whether the tensor’s buffer can be mutated in place — i.e. this handle uniquely owns it, so an in-place op (slice_assign, an inplace kernel) writes the existing allocation instead of copying it first.

fn scheme(&self) -> QuantScheme

Get the quantization scheme for a quantized float tensor.