Module loader 

Loads a real Decoder from an on-disk GGUF file, using the llama.cpp-style tensor naming convention (token_embd.weight, blk.N.attn_q.weight, blk.N.ffn_gate.weight or, for MoE, blk.N.ffn_gate_exps.weight, output_norm.weight, output.weight). Until this module existed, ferrox could only run correctly-shaped random weights.

Quantized tensors (Q8_0 / Q4_0) are loaded as WeightMatrix::Quantized backed by WeightBytes::Mapped – a zero-copy view into the same mmap GgufFile already holds, with no intermediate heap copy of the tensor’s bytes at all. So a checkpoint’s resident memory is the mmap page cache, not the mmap plus a second in-process copy of every weight. WeightMatrix::apply dispatches to ferrox-quant’s fused dequant+dot kernels directly against those mapped bytes at inference time. F32 tensors (norms, embeddings, and any weight not natively quantized) still copy into an owned Tensor, since they’re small relative to the quantized weight matrices and need per-element access patterns a raw byte view doesn’t support as cleanly.

Verified end to end (see crates/ferrox-models/tests/gguf_roundtrip.rs) against a genuinely Q8_0-quantized, generated on-disk GGUF fixture for the dense (single-expert) case, and against real OLMoE / Qwen2-MoE checkpoints for the multi-expert 3D-packed-tensor path.

Structs

GgufExpertSource

ExpertSource over a (possibly sharded) GGUF checkpoint: each expert’s gate/up/down byte ranges are read positionally from the owning shard file and concatenated, so a store miss touches exactly that expert’s bytes – no mmap of the expert region, no shared seek cursor.

StoredExpertLayout

Byte-range layout of one store-backed routed expert.

StoredMatrixSpec

One matrix’s place inside a store-backed expert’s combined byte buffer (gate bytes, then up, then down, concatenated by GgufExpertSource::read_expert).

Enums

LoadError

Functions

assert_every_tensor_consumed

Fails the load when the checkpoint carries tensors this build never looked at.