Enum QTensor 

pub enum QTensor {
    F32 {
        data: Vec<f32>,
        rows: usize,
        cols: usize,
    },
    Mapped {
        model: Arc<CmfModel>,
        idx: usize,
        dtype: TensorDtype,
        rows: usize,
        cols: usize,
        row_scale: Vec<f32>,
        col_field: Vec<f32>,
    },
}

Variants

F32

Fields

data: Vec<f32>

rows: usize

cols: usize

Mapped

Fields

model: Arc<CmfModel>

idx: usize

Index into the model’s tensor directory.

dtype: TensorDtype

rows: usize

cols: usize

row_scale: Vec<f32>

Per-row scales, dequantized to f32 up front (tiny).

col_field: Vec<f32>

q8_2f column field (θ), dequantized up front; empty for q8_row.

Implementations

impl QTensor

pub fn from_f32(data: Vec<f32>, rows: usize, cols: usize) -> Self

pub fn from_model(model: &Arc<CmfModel>, name: &str) -> Result<Self, String>

Wrap a directory tensor without dequantizing the payload. Falls back to dequantized f32 for dtypes without a fused kernel.

pub fn rows(&self) -> usize

pub fn cols(&self) -> usize

pub fn as_f32(&self) -> Option<&[f32]>

Dense f32 view — only for owned tensors. Masked/sparse execution paths require it; quantized weights don’t support masks yet.

pub fn row_f32(&self, r: usize, dst: &mut [f32])

Dequantize one row into dst (embedding lookup).

pub fn sparse_col_ok(&self) -> bool

Can this tensor’s columns be read cheaply (for sparse down_proj)? True for F32/Q8Row/Q8_2f (per-row scale, direct strided access); false for group-packed q4/vbit (column access would unpack whole groups — sparse execution falls back to f32 for those).

pub fn add_col_scaled(&self, c: usize, w: f32, out: &mut [f32])

down_proj [hidden, inter]: accumulate w · col(c) into out [hidden] — reads ONLY column c (one neuron) from the mmap, no full-matrix dequant. out[k] += w · down[k, c].

pub fn row_dot(&self, r: usize, x: &[f32], scratch: &mut [f32]) -> f32

Dot of row r with x (gate/up active-neuron path). Reads only row r from the mmap — no full dequant. q4/vbit dequant the row into scratch first (rare for active-FFN weights).

pub fn matvec(&self, x: &[f32], out: &mut [f32], pool: Option<&Pool>)

out = W · x (row-major). F32 delegates to the historical bit-exact path; Mapped runs the fused int8 kernel.

pub fn matvec2( &self, x1: &[f32], x2: &[f32], o1: &mut [f32], o2: &mut [f32], pool: Option<&Pool>, )

Fused two-input matvec (MTP verify pair): weights streamed once.

impl QTensor

pub fn matmat( &self, xs_all: &[f32], b: usize, out: &mut [f32], pool: Option<&Pool>, )

Batched matvec (prefill-GEMM): xs — row-major [b, cols], out — row-major [b, rows]. Element-wise semantics are IDENTICAL to b matvec calls (same dot kernels in the same order); the win — the weight row streams from DRAM once per batch, not b times.