Struct TiledQ4KMatvec 

pub struct TiledQ4KMatvec {
    pub config: TilingConfig,
    pub m: usize,
    pub k: usize,
}

Tiled Q4_K MatVec executor

Implements TCB-01 pattern: Cache-blocked matvec with 4×1 micro-kernel.

Memory Layout

Weights are stored in Q4_K superblock format (144 bytes per 256 elements):

• d: f16 (2 bytes) - block scale
• dmin: f16 (2 bytes) - block minimum
• scales: 12 bytes - 8 sub-block scales (6-bit packed)
• qs: 128 bytes - 256 quantized values (4-bit packed)

Performance Characteristics

• L2-resident: Process midi_tile.m rows at a time
• Vectorized: 4×1 micro-kernel processes 4 output rows simultaneously
• Aligned: K dimension aligned to Q4_K superblock (256)

Fields

config: TilingConfig

Tiling configuration

m: usize

Number of rows (M dimension)

k: usize

Number of columns (K dimension)

Implementations

impl TiledQ4KMatvec

pub fn new(m: usize, k: usize) -> Self

Create a new tiled Q4K matvec executor

Panics

Panics if K is not aligned to Q4_K superblock size (256).

pub fn superblocks_per_row(&self) -> usize

Get number of superblocks per row

pub fn total_superblocks(&self) -> usize

Get total number of superblocks

pub fn weight_row_offset(&self, row: usize) -> usize

Get weight bytes offset for a given row

pub fn optimal_parallel_rows(&self, l2_bytes: usize) -> usize

Calculate optimal number of parallel rows based on L2 cache

Goal: Keep working set in L2 (256KB typical) Working set = midi_tile.m rows × K × sizeof(Q4K) + K × sizeof(f32)

pub fn execute_scalar(&self, weights: &[u8], input: &[f32], output: &mut [f32])

Execute tiled matvec (reference scalar implementation)

This is the reference implementation for correctness testing. Actual SIMD implementation would be in the backends.

For parallel execution, use [execute_parallel] when the parallel feature is enabled.

pub fn execute_parallel( &self, weights: &[u8], input: &[f32], output: &mut [f32], )

Execute tiled matvec with parallel row processing

Uses Rayon to parallelize across rows for multi-core speedup. Falls back to scalar execution if the parallel feature is not enabled.

Performance

Achieves near-linear speedup with core count for large matrices. For small matrices (< 256 rows), scalar may be faster due to overhead.

pub fn stats(&self) -> TilingStats

Get tiling statistics for profiling

Trait Implementations

impl Clone for TiledQ4KMatvec

fn clone(&self) -> TiledQ4KMatvec

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TiledQ4KMatvec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.