Skip to main content

synth_matmul_codebook

Function synth_matmul_codebook 

Source
pub fn synth_matmul_codebook(
    x: &[f32],
    indices: &[u8],
    codebook: &[f32],
    out: &mut [f32],
    m: usize,
    k: usize,
    n: usize,
    entry_dim: usize,
)
Expand description

Codebook weight-synthesis matmul (single-level vector quantization).

The weight is stored transposed ([n, k], GGUF “bt” layout) as codebook indices: indices[j, kb] selects centroid codebook[indices[j,kb]] ∈ ℝ^{entry_dim}, which reconstructs the entry_dim weights at W[j, kb·entry_dim .. (kb+1)·entry_dim]. Output is y = x · Wᵀ, i.e. out[i, j] = Σ_p x[i, p] · W[j, p]. Like the dequant kernels, the win is weight-read bandwidth: the centroid is expanded inside the accumulate and the k·n f32 weight is never materialized (on the decode GEMV path).

Requires k % entry_dim == 0 (guaranteed by the op’s shape contract).