Skip to main content

theoretical_max_error

Function theoretical_max_error 

Source
pub fn theoretical_max_error(params: &QuantParams) -> f32
Expand description

Calculate theoretical maximum error for given quantization parameters

For symmetric quantization: max_error = scale / 2 (half quantization step) For asymmetric: max_error = scale / 2