Skip to main content

build_launch

clawft_kernel::workload_runtime::infer::launch

Function build_launch 

Source 
pub fn build_launch(
    flavor: InferFlavor,
    spec: &InferenceSpec,
    model: &ResolvedModel,
    draft: Option<&ResolvedModel>,
    cfg: &ManagedConfig,
    instance_id: &str,
) -> Result<Launch, RuntimeError>
Expand description

Build the launch for a llama.cpp or mlx-lm instance. The argument shape follows the model lab’s launchers (serve-llamacpp <model.gguf> [--draft m] [--kv q] [--ctx n] [--port p] [--host h], serve <model> --port p --host h); it lives in this one function.