Skip to main content

quantize_model_weights

sensorlm::quantization::int8

Function quantize_model_weights 

Source 
pub fn quantize_model_weights(
    config_json: String,
    layers: impl IntoIterator<Item = (String, Vec<f32>, Vec<usize>, Option<Vec<f32>>)>,
) -> QuantizedModel
Expand description

Quantise a named list of FP32 weight tensors.

This function takes the flat (name, weights, shape, bias) representation that would be extracted from a SensorLMModel and produces a QuantizedModel.

§Arguments

  • config_json – JSON string of the model config.
  • layers – Iterator of (name, fp32_weights, shape, optional_bias).