Struct CqWeight 

pub struct CqWeight {
    pub out_feat: usize,
    pub in_feat: usize,
    pub in_padded: usize,
    pub group: usize,
    pub bits: u8,
    /* private fields */
}

A Cactus-Quants weight matrix, logically [out_feat, in_feat].

Fields

out_feat: usize

in_feat: usize

Logical (unpadded) reduction width.

in_padded: usize

Reduction width rounded up to a multiple of group.

group: usize

bits: u8

Record width as stored in the directory: 2, 3, 4, or 5 for ternary.

Implementations

impl CqWeight

pub fn select_rows(&self, rows: &[usize]) -> Option<Self>

A new tensor holding only rows, in the order given.

Cactus-Quants packs each output row into its own row_bytes run with its own per-group norms, so a row is independently addressable: taking a subset is a copy, not a re-quantisation. That is what makes slicing a ladder rung out of the full model exact rather than approximate — the kept rows keep the codes they were trained with.

Returns None if any index is out of range.

pub fn from_blob( blob: &[u8], out_feat: usize, in_feat: usize, group: usize, bits: u8, codebook: &[f32], ) -> Result<Self, CqError>

Parse a CQ tensor from its blob: packed indices followed by FP16 norms.

codebook is the concatenated cb2|cb3|cb4 block from the .cact header; it is ignored for ternary, whose levels are analytic.

pub fn packed_bytes(&self) -> usize

Bytes of packed weight held (excluding norms) — for size accounting.

pub fn effective_bits(&self) -> f32

Effective bits per weight, counting the FP16 group norms.

pub fn prepared_len(&self) -> usize

Scratch length [prepare_input] needs.

pub fn prepare_input(&self, x: &[f32], xh: &mut [f32])

Rotate an activation into the domain the packed weights live in.

x is in_feat long; xh must be prepared_len long and is fully overwritten. Reusable across every CqWeight with the same in_feat and group.

pub fn matvec(&self, x: &[f32], y: &mut [f32])

y = W · x, allocating the rotation scratch.

Prefer prepare_input + matvec_prepared when several weights share one activation — a transformer layer’s q/k/v/gate projections all do.

pub fn matvec_prepared(&self, xh: &[f32], y: &mut [f32])

y = W · x given an activation already rotated by prepare_input.

pub fn matvec_rows_prepared(&self, xh: &[f32], row_start: usize, y: &mut [f32])

y = W[row_start .. row_start + y.len()] · x, on a prepared activation.

The mHC projections are stored as one tall matrix per kind, with a four-row band per layer (sixteen for phi_res), so a full matvec would compute 27x more rows than a layer needs.

pub fn matvec_prepared_reference(&self, xh: &[f32], y: &mut [f32])

Deliberately simple reference for the packed dot product.

One running scalar accumulator, no lane splitting, no byte LUT — decode each index and multiply. Kept public and tested against matvec_prepared so the optimised paths have an obviously-correct counterpart, and so the effect of the optimisation can be measured against it under identical conditions rather than against a number recorded at a different time.

Not for production use: it is several times slower.

pub fn matmul_rows_prepared( &self, xh: &[f32], batch: usize, row_start: usize, rows: usize, y: &mut [f32], acc: &mut [f32], )

y[b * rows + r] = W[row_start + r] · xh[b], over batch prepared activations laid out back to back at prepared_len stride.

This is where batching pays. A matvec streams the whole packed weight set once per position — 11 MB for the shipped model, far past any core’s private cache, so every position re-reads it from further out. Here each group is decoded once and applied to all batch activations, so the weight traffic is paid once for the batch and the inner loop is pure FMA over values already in registers.

Bit-identical to batch calls to matvec_rows_prepared, not merely close: the group norm is applied after the group dot, and the dot is summed in the same lane order, so no reassociation is introduced.

acc is scratch, at least batch long.

pub fn dequantize_row(&self, o: usize, out: &mut [f32])

Reconstruct one output row into out (in_feat long).

Used where a row is genuinely needed as a vector rather than as a dot product — token embedding lookup, and parity tests.

pub fn dequantize_to(&self, out: &mut [f32])

Reconstruct the whole matrix, row-major [out_feat, in_feat].