Skip to main content

serve_profile_runtime_entries

Function serve_profile_runtime_entries 

Source
pub fn serve_profile_runtime_entries(
    snapshot_path: &Path,
    device: &Device,
    vnext_plan_owns_context_capacity: bool,
    current: &RuntimeConfigSnapshot,
    source: RuntimeConfigSource,
) -> Vec<RuntimeConfigEntry>
Expand description

Product defaults for ferrum serve on GGUF LLMs.

GGUF paths do not go through the HF-directory autosizer. Without an explicit profile, Qwen3-30B-A3B falls back to the model default FERRUM_KV_CAPACITY=512, which can make a normal sequence of OpenAI requests (sync correctness, multi-turn, then stream) end the stream immediately with an empty EOS. Keep this product path correct by default: enough context for Qwen3 thinking-mode responses and a multi-request pool for dense and MoE GGUF serving. A registered vNext execution plan owns its context capacity through admission and the dynamic resource pool, so the legacy static capacity guard must not override it.