Skip to main content

detect_backend

Function detect_backend 

Source
pub fn detect_backend(endpoint: &str) -> Result<String>
Expand description

Blocking probe of /v1/models to detect which backend engine is running.

Inspects response headers (Server) and body content for tell-tale strings from llama.cpp, SGLang, or vLLM. Returns a lower-case label like "llama.cpp", "sglang", "vllm", or "unknown".

This is a synchronous helper so it can be called from the bench-harness runner (which runs inside spawn_blocking).