Module decoder 

Generic decoder-only transformer forward pass, assembled from a ModelConfig. Each layer is: RMSNorm -> GQA attention (+RoPE) -> residual -> RMSNorm -> MoE FFN (router + routed experts + shared experts) -> residual. This is the standard decoder block shape shared by the LLaMA/DeepSeek/GLM/Kimi family of open-weight models.

Weight loading from a real GGUF checkpoint lives in loader (Decoder::from_gguf); Decoder::new_random builds correctly-shaped, randomly initialized weights so the full pipeline – embedding lookup, N decoder layers, output head – can be exercised end to end with real assertions about shapes, finiteness, and determinism, without requiring a multi-hundred-gigabyte checkpoint to be present.

Structs

AttnWeights

Decoder

GptOssLayer

The per-layer weights the gpt-oss graph carries and the generic GQA layer structs do not.

GptOssWeights

gpt-oss side table: one entry per layer, in layer order.

LayerWeights

MoeWeights

Enums

ExpertBacking

How a layer’s routed experts are held. Resident is the original always-in-memory form (owned f32 or zero-copy mmap views). Stored holds only byte-range layouts; each use acquires the expert’s bytes from a bounded, lease-protected ferrox_core::expert_store::ExpertStore shared by every layer (one global byte budget), builds temporary WeightMatrix views over the leased buffer (WeightBytes::Shared, which pins the cache entry for the views’ lifetime), and drops them after the expert runs. Dequantized math over identical bytes is identical, so the two backings are bit-equivalent by construction – pinned by an integration test against the MoE fixture.

MultiSeqKv

Where a batch of independent sequences keeps its KV.