Struct AwqHostWeight 

pub struct AwqHostWeight {
    pub weight: Arc<[u32]>,
    pub zeros: Arc<[u32]>,
    pub scales: Arc<[f16]>,
    pub zeros_w: usize,
    pub oc: usize,
    pub ic: usize,
    pub group_size: usize,
}

An AWQ linear weight as host buffers in the kernel layout, bound to no graph: what a model’s weight set stores (WeightStorage::Awq) so the same weights can be placed on any graph or device. The matmul builds its constants on the activations’ graph.

Fields

weight: Arc<[u32]>

[OC, IC/8] U32, sequential nibble order.

zeros: Arc<[u32]>

[OC, zeros_w] U32.

scales: Arc<[f16]>

[OC, 8 * zeros_w] F16.

zeros_w: usize

Words per zeros row.

oc: usize

Output features.

ic: usize

Input features.

group_size: usize

Quantization group size (64 or 128).

Implementations

impl AwqHostWeight

pub fn from_hf_buffers( qweight: &[u32], qzeros: &[u32], scales: &[f16], ic: usize, oc: usize, group_size: usize, ) -> Result<Self>

Repack HF-layout buffers (qweight [IC, OC/8], qzeros [IC/g, OC/8], scales [IC/g, OC]) into the kernel layout.

pub fn load<'a>( get: impl Fn(&str) -> Result<TensorView<'a>>, prefix: &str, ic: usize, oc: usize, group_size: usize, ) -> Result<Self>

Read one WQLinear_GEMM layer ({prefix}.qweight/.qzeros/.scales) through get, validating every dtype and shape against the geometry.

pub fn matmul(&self, activations: &Tensor, split_k_iters: i32) -> Result<Tensor>

activations @ dequant(weight)ᵀ via the fused AWQ_MATMUL op, with the weight constants built on activations’ graph. Output [..., M, OC] in the activations’ dtype. split_k_iters must be positive (see AwqWeight::matmul).

Trait Implementations

impl Clone for AwqHostWeight

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AwqHostWeight

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.