Struct AwqWeight 

pub struct AwqWeight {
    pub weight: Tensor,
    pub zeros: Tensor,
    pub scales: Tensor,
    pub oc: usize,
    pub ic: usize,
    pub group_size: usize,
}

An AWQ-quantized linear weight, materialized as the three Tensor inputs fuel_graph::NodeHandle::awq_matmul expects (kernel layout), plus the geometry the matmul builder needs.

Fields

weight: Tensor

[OC, IC/8] U32, sequential nibble order.

zeros: Tensor

[OC, zeros_w] U32.

scales: Tensor

[OC, 8 * zeros_w] F16.

oc: usize

Output features.

ic: usize

Input features.

group_size: usize

Quantization group size (64 or 128).

Implementations

impl AwqWeight

pub fn matmul(&self, activations: &Tensor, split_k_iters: i32) -> Result<Tensor>

activations @ dequant(weight)ᵀ via the fused AWQ_MATMUL op. activations must live on the same graph as the weight tensors (build it from self.weight.graph_tensor()). Output [..., M, OC] in the activations’ dtype.

split_k_iters is a CUDA launch-tuning parameter (8 is AutoAWQ’s default); the decompose path ignores it, so it is validated here rather than left to a kernel launch: it must be positive. The kernel’s own constraints (ic % (32 * split_k_iters) == 0, oc % 64 == 0, sm_80+) are baracuda’s can_implement’s to enforce at launch.