Enum CpuQuantStore 

pub enum CpuQuantStore {
    Q4K {
        weights: Vec<f32>,
        n_rows: usize,
        n_cols: usize,
    },
}

CPU-side container for any GGUF k-quant flavour. Each variant holds the dense fp32 weights post-eager-dequant — CPU isn’t the bench target so we don’t pay the complexity of on-the-fly dequant here; the variant tag exists so gemm_quant can route consistently.

New k-quant types (Q5_K / Q6_K / Q8_0) become new variants — no trait churn, just a new arm in load_quant and gemm_quant.

Variants

Q4K

Fields

weights: Vec<f32>

n_rows: usize

n_cols: usize