pub fn enhance_body_for_prefetch(
base_url: Option<&str>,
body: &mut Map<String, Value>,
)Expand description
Enhances an OpenAI-compatible request body with prefetch/caching hints based on backend.
Currently, for llama.cpp HTTP server (commonly http://127.0.0.1:8080), we set cache_prompt = true
to enable prompt prefill caching across related requests.