Skip to main content

quantization_error_analysis

Function quantization_error_analysis 

Source
pub fn quantization_error_analysis(
    original: &Tensor,
    quantized: &Tensor,
) -> Result<(f32, f32, f32)>
Expand description

Quantization error analysis

Analyzes the error introduced by quantization to guide optimization.

§Arguments

  • original - Original floating-point tensor
  • quantized - Quantized tensor (after dequantization)

§Returns

Tuple of (mse_error, max_error, snr_db)