Struct QuantizedLinear 

pub struct QuantizedLinear {
    pub in_features: usize,
    pub out_features: usize,
    pub quant_type: QuantType,
    /* private fields */
}

A linear layer with quantized weights for fast inference.

Stores weights as QuantizedTensor (Q8/Q4/F16) and dequantizes on-the-fly during the matrix multiply. Bias remains in f32.

Usage

use axonml_quant::inference::QuantizedLinear;
use axonml_quant::QuantType;

let qlinear = QuantizedLinear::from_linear_params(&weights, Some(&bias), 512, 128, QuantType::Q8_0);
let output = qlinear.forward_f32(&input_data, 1);

Fields

in_features: usize

Input feature dimension.

out_features: usize

Output feature dimension.

quant_type: QuantType

Quantization type used.

Implementations

impl QuantizedLinear

pub fn from_linear_params( weight_data: &[f32], bias_data: Option<&[f32]>, in_features: usize, out_features: usize, quant_type: QuantType, ) -> Self

Create a QuantizedLinear from an axonml_nn::Linear layer.

pub fn forward_f32(&self, input: &[f32], batch_size: usize) -> Vec<f32>

Forward pass: f32 input → f32 output.

Input shape: [batch, in_features] Output shape: [batch, out_features]

Performs quantized matrix multiplication: each output element is computed by iterating over the weight row’s quantized blocks and accumulating dot products with the corresponding activation slices.

pub fn forward_var(&self, input: &Variable) -> Variable

Forward pass with Variable input/output (for integration with autograd).

Note: Quantized inference is forward-only (no gradient tracking). The output Variable has requires_grad = false.

pub fn weight_bytes(&self) -> usize

Memory usage in bytes (weights only, excludes bias).

pub fn compression_ratio(&self) -> f32

Compression ratio vs f32 weights.

pub fn dequantize_weights(&self) -> Tensor<f32>

Dequantize weights back to f32 (for debugging/validation).

Trait Implementations

impl Clone for QuantizedLinear

fn clone(&self) -> QuantizedLinear

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantizedLinear

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.