Struct QuantScheme 

pub struct QuantScheme {
    pub value: QuantValue,
    pub store: QuantStore,
    pub mode: QuantMode,
    /* private fields */
}

Describes a quantization scheme/configuration.

Scales come at up to two levels, each an optional field set through per_tensor and per_block in any order:

// One scale for the whole tensor, stored as f32. Also what a scheme with no level resolves to.
QuantScheme::default().per_tensor(ScaleDtype::F32);

// One scale per block of 32 values.
QuantScheme::default().per_block([32], ScaleDtype::F32);

// Two levels: ue4m3 block scales, normalized by a single per-tensor f32 scale.
QuantScheme::default()
    .per_block([16], ScaleDtype::UE4M3)
    .per_tensor(ScaleDtype::F32);

A two-level scheme exists so block scales can live in a narrow type: the global per-tensor scale absorbs the tensor’s dynamic range, and the block dtype only covers the spread between blocks. That spread is still bounded: a block whose scale falls further below the largest one than the block dtype can express is stored at that dtype’s smallest value, far too coarse for it, and every value in the block quantizes to zero. ScaleDtype::UE4M3 spans about 2^18 this way, so a tensor holding a genuine outlier can lose its ordinary values.

Fields

value: QuantValue

The logical data type of quantized input values (e.g., QuantValue::Q8F).

This defines how values are interpreted during computation, independent of how they’re stored.

store: QuantStore

Data type used for storing quantized values.

mode: QuantMode

Quantization mode (e.g., symmetric).

Implementations

impl QuantScheme

pub fn nvfp4() -> Self

The NVFP4 format: fp4 (e2m1) values in blocks of 16, with ue4m3 block scales normalized by one per-tensor f32 scale.

impl QuantScheme

pub fn with_mode(self, mode: QuantMode) -> Self

Set the quantization mode.

pub fn with_value(self, value: QuantValue) -> Self

Set the data type used for quantized values.

pub fn with_store(self, store: QuantStore) -> Self

Set the data type used to store quantized values.

pub fn per_tensor(self, dtype: ScaleDtype) -> Self

Set the per-tensor scale level, stored as dtype.

pub fn per_block(self, block: impl AsRef<[u8]>, dtype: ScaleDtype) -> Self

Set the per-block scale level: one scale per block of block values, stored as dtype.

pub fn tensor_scale(&self) -> Option<ScaleDtype>

The per-tensor scale level, the global level when a block level is present.

A scheme storing no level at all resolves here to a per-tensor f32 scale; the resolution is not stored, so such a scheme compares equal to Default, not to an explicit per_tensor(F32).

pub fn block_scale(&self) -> Option<BlockScale>

The per-block scale level, the innermost when both levels are present.

pub fn num_levels(&self) -> usize

The number of scale levels: as many scale tensors ride along with the values.

pub fn scale_dtype(&self) -> ScaleDtype

The innermost level’s scale dtype, the type the per-position scales are stored in.

pub fn block_size(&self) -> Option<BlockSize>

The block level’s size, or None for per-tensor quantization.

pub fn swap_block_dims(&mut self, rank: usize, dim0: usize, dim1: usize)

Swap two tensor dimensions in the block level, mirroring shape.swap(dim0, dim1). The per-tensor level is unaffected.

dim0/dim1 are bare indices on purpose, mirroring [T]::swap’s own signature.

pub fn permute_block_dims(&mut self, rank: usize, axes: &[usize])

Permute the block level, mirroring a permutation of the tensor’s axes. The per-tensor level is unaffected.

pub fn size_bits_stored(&self) -> usize

Returns the size of the quantization storage type in bits.

pub fn size_bits_value(&self) -> usize

Returns the size of the quantization storage type in bits.

pub fn num_quants(&self) -> usize

Returns the number of quantized values stored in a single element.

pub fn native_packing(&self) -> usize

Returns the native packing factor for the values. When native packing > 1, the packed representation stores num_quants elements grouped into packs of native_packing size.

pub fn packing_dim(&self) -> Option<usize>

Returns the packing dim for the store.

pub fn swap_packing_dim(&mut self, dim0: usize, dim1: usize)

Swaps the packing dim if it’s either of dim0 or dim1. Executes the corresponding update to shape.swap(dim0, dim1).

Trait Implementations

impl Clone for QuantScheme

fn clone(&self) -> QuantScheme

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for QuantScheme

impl Debug for QuantScheme

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for QuantScheme

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for QuantScheme

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Eq for QuantScheme

impl Hash for QuantScheme

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl Ord for QuantScheme

fn cmp(&self, other: &QuantScheme) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

where Self: Sized,

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

where Self: Sized,

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

where Self: Sized,

Restrict a value to a certain interval.

impl PartialEq for QuantScheme

fn eq(&self, other: &QuantScheme) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl PartialOrd for QuantScheme

fn partial_cmp(&self, other: &QuantScheme) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

Tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

Tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

Tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

Tests greater than or equal to (for self and other) and is used by the >= operator.

impl Serialize for QuantScheme

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for QuantScheme