Expand description
OpenAI Chat Completions backend with SSE streaming.
Covers every OpenAI-compatible endpoint: OpenAI itself plus DeepSeek,
Groq, Cerebras, xAI, Mistral, Gemini (OpenAI-compat path), Together,
Fireworks, OpenRouter, NVIDIA NIM, Moonshot/Kimi, ZAI, MiniMax, Qwen
(DashScope compat), HuggingFace router, Vercel AI Gateway, OpenCode Zen,
Azure OpenAI, Ollama / LM Studio / vLLM, and any custom
INFINITY_OPENAI_BASE_URL proxy.
Structs§
- Backend
- How to reach and authenticate one
OpenAI-compatible backend.
Functions§
- build_
body - Build a non-streaming request body (used by tests).
- discover_
models - Probe an OpenAI-compatible
GET {base}/modelsendpoint. - discovered_
ids - Extract model ids from an OpenAI-style
GET /modelsbody. - invoke
- Stream one completion from an
OpenAI-compatible endpoint.