Skip to main content

Module quantization

singe_kernel::cpu

Module quantization 

Source
Expand description

Quantize, dequantize, int4 unpacking, fp8 helpers, and quantized matmul.

Structs§

DequantizedWeightMatmulConfig
QuantizedMatmulConfig

Functions§

dequantize_f8e4m3_block_f32
dequantize_i8_grouped
dequantize_u8_grouped
f8e4m3_value
gather_strided_f32
matmul_f8e4m3_block_scaled_f32
matmul_f32_i8_dequantize_f32
matmul_i8_i8_f32
scatter_strided_f32
scatter_strided_i8