Struct QuantizedModel 

pub struct QuantizedModel {
    pub quantized_params: Vec<QuantizedTensor>,
    pub quant_type: QuantType,
    pub total_params: usize,
    pub total_bytes: usize,
    pub original_bytes: usize,
}

A fully quantized model for inference.

Wraps a collection of quantized parameters and provides fast forward pass by dequantizing weights on-the-fly during computation.

Usage

use axonml_quant::inference::QuantizedModel;
use axonml_quant::QuantType;

let qmodel = QuantizedModel::from_module(&model, QuantType::Q8_0);
println!("{}", qmodel.summary());
qmodel.load_into_module(&model); // dequant weights back for inference

Fields

quantized_params: Vec<QuantizedTensor>

Quantized weight tensors (in parameter order).

quant_type: QuantType

Quantization type used.

total_params: usize

Total original parameter count.

total_bytes: usize

Total quantized size in bytes.

original_bytes: usize

Original f32 size in bytes.

Implementations

impl QuantizedModel

pub fn from_module<M: Module>(module: &M, quant_type: QuantType) -> Self

Quantize a Module’s parameters.

pub fn load_into_module<M: Module>(&self, module: &M)

Load quantized weights back into a Module for inference.

Dequantizes all parameters and updates the module’s parameters in-place. This is the simplest integration path — the model runs at full f32 speed but loads from a compressed checkpoint.

pub fn compression_ratio(&self) -> f32

Compression ratio.

pub fn summary(&self) -> String

Print a summary of the quantized model.