Struct QuantizedLinear 

pub struct QuantizedLinear<B: Backend> { /* private fields */ }

y = x @ W^T (+ b) where W is stored group-quantized to 4 bits.

Implementations

impl<B: Backend> QuantizedLinear<B>

pub fn new( packed: Tensor<B, 2, Int>, scales: Tensor<B, 2>, group_size: usize, bias: Option<Tensor<B, 1>>, ) -> Result<Self>

Builds a layer from on-device packed parts, validating shapes.

pub fn in_features(&self) -> usize

Input feature count (dequantized in_features).

pub fn out_features(&self) -> usize

Output feature count.

pub fn weight(&self) -> Tensor<B, 2>

Dequantizes the weight to f32 [out_features, in_features].

pub fn forward(&self, x: Tensor<B, 3>) -> Tensor<B, 3>

y = x @ dequant(W)^T (+ b) for x: [batch, seq, in_features].