Expand description
QTensor — weight tensor with pluggable storage.
Two backings, one interface:
F32— owned dense floats (small models, tests). Every operation is bit-identical to the historical&[f32]code paths.Mapped— quantized bytes zero-copy from the CMF mmap (q8_row/q8_2f). The matvec is fused: int8 rows × f32 activations, the q8_2f column field folds into a pre-scale of the input (x'[i] = col[i]·x[i]), so the inner loop is the same i8 dot as q8_row. This is what lets a 15B file run in a few GB of RSS.
Extension point: new dtypes = new match arm here, nothing else moves.