Struct HostQTensor 

pub struct HostQTensor { /* private fields */ }

Quantized tensor for the Flex backend.

Stores quantized i8 values in the tensor and keeps scales separately for efficient dequantization without reparsing bytes.

Implementations

impl HostQTensor

pub fn new(tensor: HostTensor, scheme: QuantScheme, scales: Vec<f32>) -> Self

Create a new quantized tensor.

The tensor must store i8 data and scales must match the quantization parameter shape.

pub fn tensor(&self) -> &HostTensor

Get the underlying tensor.

pub fn scales(&self) -> &[f32]

Get the quantization scales.

Trait Implementations

impl Clone for HostQTensor

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for HostQTensor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl QTensorPrimitive for HostQTensor

fn scheme(&self) -> &QuantScheme

Returns the quantization settings for the given tensor.

fn default_scheme() -> QuantScheme

Returns the default tensor quantization scheme.

fn acc_precision(&self) -> QuantAcc

The precision used for the accumulation in various kernels.

fn propagation(&self) -> QuantPropagation

How quantization is propagated during computation.

impl TensorMetadata for HostQTensor

fn dtype(&self) -> DType

The dtype of the tensor.

fn shape(&self) -> Shape

The shape of the tensor.

fn rank(&self) -> usize

The number of dimensions of the tensor.