Module engine 

A trait-based abstraction over the two structurally different but text-in/text-out-shaped forward passes this crate has: Decoder (GQA+RoPE, used by GLM-5.2/DeepSeek V4 Pro and every real GGUF checkpoint) and Kimi K3’s dedicated hybrid KDA/Gated-MLA stack (kimi_decoder). This lets ferrox-server share one generic generation loop across both engines (see ferrox-server::generate::generate_engine) for the actual sampling/stop-sequence logic, rather than hand-duplicating it – while keeping GGUF-only features (the KV block pool, PrefixCache) as Decoder-specific code layered on top, not forced into this trait. The reason: Kimi’s KDA state is a fixed-size recurrent matrix that collapses history irreversibly, so it cannot support the same restore/truncate operations KvCache can – unifying those too would mean either a leaky abstraction or silently pretending Kimi supports something it doesn’t.

forward_token’s pos parameter is meaningful for Decoder (used directly for RoPE) but not for KimiEngine: Kimi’s real forward pass (kimi_forward_token) derives position purely from its own per-layer state (KDA’s recurrent state, MLA’s growing K/V buffers) – its real signature has no pos parameter at all. KimiEngine ignores the argument; this is a real architectural fact about the model, not an oversight in this trait’s design.

Structs

DeepseekV4Engine

DeepSeek V4 synthetic stack behind Engine. Preset deepseek_v4_pro remains a sketch until a real GGUF loader + incremental DSV4 KV land.

Glm52Engine

GLM-5.2 dedicated DSA stack behind the same Engine trait as Kimi. Synthetic / loader-backed weights only — no claim of a full real ~744B serve path. Lets generate_engine exercise GLM without forcing DSA into the GQA Decoder.

KimiEngine

Bundles Kimi K3’s weights with the three real config structs kimi_forward_token needs, so Engine::forward_token’s three- argument shape (token_id, pos, state) can wrap Kimi’s real four-config-argument function.

MlaDecodeState

MlaDenseFfn

MlaEngine

Multi-layer MLA stack for DeepSeek-2 / Mistral-4-style GGUF serve.

MlaLayerWeights

MlaMoeFfn

MlaMoeRuntime

MoE routing knobs shared by every MoE layer (DeepSeek-2 / Mistral-4).

Enums

MlaLayerFfn

Traits

Engine

A decoder that can run one incremental forward step given a token id and position, updating its own per-layer state in place.

TextTokenizer

A minimal text<->token-id interface shared by every real tokenizer this crate has, regardless of each one’s native id width (GgufBpeTokenizer/GgufSpmTokenizer/GgufUnigramTokenizer use u32, KimiTokenizer also uses u32) – lets a generic generation loop encode/decode without caring which concrete tokenizer it was given.