Struct QMmap 

pub struct QMmap<T> { /* private fields */ }

A CPU quantized store whose blocks live in an mmap’d GGUF region rather than an owned Vec.

Holds an Arc<memmap2::Mmap> plus the byte offset and block count of one tensor inside it, and hands the dequant/matmul path a &[T] that points directly into the mapped file – so the quantized weight bytes are never copied resident; the OS pages them in on access and reclaims them under memory pressure (page cache = “RAM as a speed spectrum”, antirez ds4_ssd). The Arc keeps the mapping alive for as long as any tensor references it.

This is the no-copy twin of QuantizedType for Vec<T>: every method forwards the same k_quants routine over self.as_slice() instead of over a Vec. It is read-only – the two from_float* (quantize) entry points are unreachable in the load path and panic if ever called (you cannot quantize into a memory-mapped, read-only weight file).

Trait Implementations

impl<T: GgmlType + Send + Sync> QuantizedType for QMmap<T>

fn matmul_t( &self, mkn: (usize, usize, usize), lhs: &[f32], dst: &mut [f32], ) -> Result<()>

fn matmul_t_f16( &self, mkn: (usize, usize, usize), lhs: &[f16], dst: &mut [f16], ) -> Result<()>

fn size(&self) -> usize

fn from_float(&mut self, _xs: &[f32])

fn from_float_imatrix( &mut self, _xs: &[f32], _imatrix_weights: &[f32], _n_per_row: usize, )

fn dtype(&self) -> GgmlDType

fn block_size(&self) -> usize

fn dequantize(&self, elem_count: usize) -> Result<CpuStorage>

fn storage_size_in_bytes(&self) -> usize

fn as_ptr(&self) -> *const u8