Skip to main content

quantize_4bit

trustformers_core::quantization

Function quantize_4bit 

Source 
pub fn quantize_4bit(
    tensor: &Tensor,
    config: &BitsAndBytesConfig,
) -> Result<QuantState>
Expand description

4-bit quantization (NF4/FP4)