Enum QuantLevel 

pub enum QuantLevel {
    Tensor,
    Block(BlockSize),
    BlockTensor {
        block: BlockSize,
        global: QuantParam,
    },
}

Level or granularity of quantization.

Append new variants, never insert. Some transports serialize this with a format that encodes variants by position rather than by name, so inserting one silently reinterprets streams and stored schemes written by an older build.

Variants

Tensor

Quantize the whole tensor using a single tensor.

Block(BlockSize)

Quantize a tensor using multiple blocks.

BlockTensor

Quantize a tensor using multiple blocks whose scales are themselves normalized by a single per-tensor scale.

See QuantLevel::block_tensor for what that buys and what it does not.

Fields

block: BlockSize

Size of each block. The block scales use QuantScheme::param.

global: QuantParam

Precision of the per-tensor scale. Only a param with more range than the block scales use is meaningful.

Implementations

impl QuantLevel

pub fn block(values: impl AsRef<[u8]>) -> QuantLevel

Converting constructor for QuantLevel::Block

pub fn block_tensor(values: impl AsRef<[u8]>, global: QuantParam) -> QuantLevel

Converting constructor for QuantLevel::BlockTensor.

The per-tensor scale absorbs the tensor’s dynamic range, which is what lets the block scales live in a narrow type. Without it a block scale has to cover that range on its own, and a type like QuantParam::UE4M3 underflows to zero for small values.

What the block param covers is then the spread between blocks, which is still bounded. A block whose scale falls further below the largest one than the block param can express is stored at that param’s smallest value, far too coarse for it, and every value in the block quantizes to zero. QuantParam::UE4M3 spans about 2^18 this way, from its smallest subnormal to 448, so a tensor holding a genuine outlier can lose its ordinary values.

The kernels in cubecl-std do not implement this level yet and reject it at launch rather than reconstruct values short by the per-tensor factor.

pub fn block_size(&self) -> Option<BlockSize>

The block size, for the levels that quantize in blocks.

pub fn global_param(&self) -> Option<QuantParam>

The precision of the per-tensor scale, for the levels that have one.

Trait Implementations

impl Clone for QuantLevel

fn clone(&self) -> QuantLevel

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for QuantLevel

impl Debug for QuantLevel

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for QuantLevel

fn deserialize<__D>( __deserializer: __D, ) -> Result<QuantLevel, <__D as Deserializer<'de>>::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Eq for QuantLevel

impl Hash for QuantLevel

fn hash<__H>(&self, state: &mut __H)

where __H: Hasher,

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl Ord for QuantLevel

fn cmp(&self, other: &QuantLevel) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

where Self: Sized,

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

where Self: Sized,

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

where Self: Sized,

Restrict a value to a certain interval.

impl PartialEq for QuantLevel

fn eq(&self, other: &QuantLevel) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl PartialOrd for QuantLevel

fn partial_cmp(&self, other: &QuantLevel) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

Tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

Tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

Tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

Tests greater than or equal to (for self and other) and is used by the >= operator.

impl Serialize for QuantLevel

fn serialize<__S>( &self, __serializer: __S, ) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for QuantLevel