Struct CmfModel 

pub struct CmfModel {
    pub path: PathBuf,
    pub header: CmfHeader,
    pub required_features: u32,
    pub tensors: Vec<TensorEntry>,
    pub masks: MaskCatalog,
    pub sparse_index: Vec<SparseIndexEntry>,
    pub vocab: Option<Vec<u8>>,
    /* private fields */
}

A loaded CMF model: metadata owned, weights zero-copy via mmap.

Fields

path: PathBuf

header: CmfHeader

required_features: u32

tensors: Vec<TensorEntry>

masks: MaskCatalog

sparse_index: Vec<SparseIndexEntry>

vocab: Option<Vec<u8>>

Embedded tokenizer.json bytes, if present.

Implementations

impl CmfModel

pub fn uid(&self) -> u64

A key that is unique to this open of the file and never recycled. Backends that cache anything derived from the weights must key on this, not on the mapping’s address, which the allocator reuses.

pub fn open(path: impl AsRef<Path>) -> Result<Self, CmfError>

Open and strictly validate a CMF v2 file. Any inconsistency is an error — this function never substitutes defaults.

pub fn open_sharded(path: impl AsRef<Path>) -> Result<Self, CmfError>

Open a sharded model (spec §10): pass shard 1; siblings found by the -{no:05}-of-{count:05}.cmf convention. Directories merge; masks/vocab/index/skills come from shard 1.

pub fn arch(&self) -> &ModelArch

pub fn tensor(&self, name: &str) -> Option<&TensorEntry>

pub fn tensor_index(&self, name: &str) -> Option<usize>

Directory index of a tensor by name (same resolution as Self::tensor — engines must not re-scan the directory). O(1) via the name-hash index; the name is verified against the entry so a hash collision can never return the wrong tensor, and the rare distinct-name collision falls back to the tiny overflow list.

pub fn resolve_tensor( &self, name: &str, skill: Option<&str>, ) -> Option<&TensorEntry>

Tensor-source indirection (spec §9, Patent 15 fig3/302): the skill’s replacement is read IN PLACE OF the backbone tensor — either/or, never combined. None skill → backbone directly.

pub fn skill_tensors( &self, skill_id: &str, ) -> impl Iterator<Item = &TensorEntry>

The per-skill delta index view (claim 2): directory entries of one skill — exactly the byte ranges lazy loading pages in.

pub fn tensor_bytes(&self, name: &str) -> Result<&[u8], CmfError>

Zero-copy bytes of a tensor from the mmap’d data section.

pub fn dir_hash(&self) -> u64

All bytes of the primary mapping (GPU path: no-copy Metal buffer over the same mmap — unified memory, zero copying). hash64 of this file’s tensor directory — the identity a standalone skill binds to (SkillRecord.base_dir_hash).

pub fn primary_bytes(&self) -> &[u8]

pub fn advise_done(&self, pred: impl Fn(&str) -> bool) -> usize

Absolute offset of the tensor within the primary mapping (None for tensors from sibling shards). Best-effort page-cache release for every tensor whose name passes pred (unix, primary shard only): the merged ranges are madvised DONTNEED so a one-shot stage’s weights — a prompt encoder that runs once per generation — stop competing for RAM with the stages after it. A 25.7 GB fl2va file on a 24 GB Mac spent 40 minutes paging the SSD during denoise for exactly this reason. Re-reading a dropped range later just refaults from disk. Returns the bytes released.

pub fn entry_abs_offset(&self, entry: &TensorEntry) -> Option<usize>

pub fn entry_bytes(&self, entry: &TensorEntry) -> &[u8]

pub fn evict_ranges(&self, ranges: &[(usize, usize)])

The CPU is done with these byte ranges of the primary mapping (absolute offsets, as entry_abs_offset hands them out): drop them from the resident set and let the page cache release the file pages. Both calls are advisory and the mapping stays valid — a range that gets touched again re-faults from disk, so a caller can only cost time here, never correctness. Ranges are aligned OUTWARD to page boundaries; the neighbours those edges claw in re-fault the same way. Linux + mmap backing only; everywhere else a no-op.

pub fn layer_tensors(&self, layer_idx: usize) -> Vec<&TensorEntry>

Tensors belonging to layer i (prefix model.layers.{i}.).

pub fn total_param_count(&self) -> u64

Total parameter count estimated from matrix tensors (ndim ≥ 2).

pub fn recode_entries_in_place( path: &str, patches: &[(usize, TensorDtype, Vec<u8>)], ) -> Result<(), CmfError>

Recode selected tensors IN PLACE: each new payload must fit its old slot, the entry keeps its offset and the file keeps its length — the bytes between the new end and the old simply go dark (every reader walks the directory, nothing addresses the gap). This is what lets a published 100+ GB file change a tensor’s layout on a disk too small to hold two copies of it. Patches are (directory index, new dtype, new payload); entry hashes and the directory hash are recomputed so verify stays clean. Not atomic: a crash between the payload writes and the directory write leaves the old dtype over new bytes — verify (or re-fetch the source) after an interrupted run.

pub fn verify(&self) -> Vec<String>

Recompute all tensor hashes; returns human-readable problems (empty = file intact).

pub fn compute_active_size(&self, mask: &TaskMask) -> u64

Approximate active weight bytes under a mask, from real tensor sizes in the directory (not from a formula).

pub fn write( path: impl AsRef<Path>, header: &CmfHeader, tensors: &[TensorSpec], masks: Option<&MaskCatalog>, vocab: Option<&[u8]>, ) -> Result<(), CmfError>

Write a CMF v2 file. Offsets, alignment, hashes and the sparse index are computed here — the caller supplies content only.

pub fn write_ref( path: impl AsRef<Path>, header: &CmfHeader, tensors: &[TensorSpecRef<'_>], masks: Option<&MaskCatalog>, vocab: Option<&[u8]>, ) -> Result<(), CmfError>

write with BORROWED tensor payloads — repack tools slice the source file’s mmap directly, so a 19 GB container rewrites without materializing its tensors in RAM (the OS streams pages through).

Trait Implementations

impl Debug for CmfModel

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.