Enum BatchActs 

pub enum BatchActs {
    Q8 {
        acts: Vec<Q8Activations>,
        tiles: Vec<Q8ActsX4>,
        cols: usize,
    },
    Q8K {
        acts: Vec<Q8KActivations>,
        tiles: Vec<Q8KActsX4>,
        cols: usize,
    },
}

A batch of activations quantized once for reuse across several WeightMatrix::apply_batch_with_acts calls that read the same input (q/k/v on one normed batch; gate/up on another). Build with WeightMatrix::quantize_batch_acts. Q8_0/Q4_0 matrices consume BatchActs::Q8; the K-quants consume BatchActs::Q8K.

tiles carries the interleaved activation quads the i8mm GEMMs read (llama.cpp’s wdata after ggml_quantize_mat_q8_K_4x8), not just the per-position quantization. Sharing stops at the same place the quantization does: q/k/v build one set between them instead of three, gate/up one instead of two. It is empty on hosts with no i8mm kernel, where preparing a quad buys nothing.

cols is recorded so a set built for one width can never be handed to a matrix of another. The tiles are chunked four positions wide for every kind (Q8K_ACTS_X4_NC, and Q4_KX8_GEMM_NC / Q5_KX8_GEMM_NC are the same 4), which is why one set serves Q4_K, Q5_K and Q6_K – and, in the BatchActs::Q8 variant, both Q8_0 and Q4_0.

Variants

Q8

Fields

acts: Vec<Q8Activations>

tiles: Vec<Q8ActsX4>

cols: usize

Q8K

Fields

acts: Vec<Q8KActivations>

tiles: Vec<Q8KActsX4>

cols: usize