Module quant

Expand description

Quantization functionality required in views

Modules§

RunWithQuantType: Run an arbitrary function with the quantization types from the scheme. Useful when concrete types aren’t available.

check_global_levels: The global-level half of check_scale_bindings, for a consumer holding global scales already folded into a register rather than as countable bindings.
check_scale_bindings: Panic when the scale bindings and the scheme’s levels disagree.
check_table_bindings: Panic when the lookup-table binding and the scheme disagree.
dequantize_aligned: Dequantize a vector of values, where vector_size * num_quants is a power of two. Unaligned values can’t be dequantized in place.
dequantize_aligned_wide: [dequantize_aligned] for a scale a caller folded in f32, forming the product there too and narrowing only the result.
multiply_global_scale: The effective scale of values whose per-tensor scale multiplies on top of their block scale.
unpack_cast_u32: Unpack a set of values from u32, and convert to the specified floating point format. table decodes each field under QuantMode::Lookup and must be None otherwise ([dequantize_aligned] checks the pairing).
unpack_fields: Unpack NF consecutive fields of one u32 word starting at field first (a runtime index), cast but unscaled — the caller multiplies by whatever scale its lines carry. The sub-word counterpart of [unpack_cast_u32], for reads whose served line is narrower than a word: NF may be any divisor of the packing factor, and first selects which slice of the word this line is. e2m1 is refused — its native pairs cannot be split at a field boundary.