Skip to main content

fake_quant_fp8

optirs_gpu::quantization

Function fake_quant_fp8 

Source 
pub fn fake_quant_fp8(
    x: &Array1<f64>,
    format: Fp8Format,
    mode: RoundingMode,
    rng: &mut impl Rng,
) -> Array1<f64>
Expand description

Fake-quantize a 1-D tensor onto an fp8 (E4M3 / E5M2) grid.

The cast is saturating: magnitudes above the format max (and infinities) clamp to ±max_normal; NaN inputs propagate to NaN. The sign of the input (including signed zero) is preserved.