Skip to main content

Module llm

ai_memory

Module llm 

Source
Expand description

LLM client — provider-agnostic chat + embedding surface.

§Providers (#1066)

The historical client was Ollama-only. Post-#1066 the same struct supports two wire shapes and any vendor that speaks either:

VariantWire shapeAuthVendors
LlmProvider::OllamaPOST /api/chat, POST /api/embednoneOllama (native)
LlmProvider::OpenAiCompatiblePOST /v1/chat/completions, POST /v1/embeddingsAuthorization: Bearer <key>OpenAI, xAI Grok, Anthropic (via OpenAI shim), Google Gemini (/v1beta/openai), DeepSeek, Kimi (Moonshot), Qwen (Alibaba), Mistral, Groq, Together AI, Cerebras, OpenRouter, Fireworks, LMStudio, vLLM, llama.cpp server, …

§Operator configuration

  • AI_MEMORY_LLM_BACKEND — selector. Accepted values:
    • ollama (default; backward compat)
    • openai-compatible — generic; requires AI_MEMORY_LLM_BASE_URL set explicitly
    • alias values that pre-fill AI_MEMORY_LLM_BASE_URL for known vendors: xai, openai, anthropic, gemini, deepseek, kimi, qwen, mistral, groq, together, cerebras, openrouter, fireworks, lmstudio
  • AI_MEMORY_LLM_BASE_URL — overrides the default per-backend URL.
  • AI_MEMORY_LLM_API_KEY — Bearer auth secret for OpenAI-compatible backends. Some aliases also accept per-vendor env vars as a convenience (e.g. XAI_API_KEY if backend=xai, OPENAI_API_KEY if backend=openai, ANTHROPIC_API_KEY if backend=anthropic, GEMINI_API_KEY if backend=gemini, etc.).
  • AI_MEMORY_LLM_MODEL — model name passed through verbatim. The selection is vendor-specific (e.g. grok-4 for xAI, deepseek-chat for DeepSeek, qwen-max for Qwen).
  • Legacy OLLAMA_BASE_URL is still honored when backend=ollama.

Structs§

OllamaClient

Enums§

LlmProvider
LLM-provider wire-shape selector. Owned by OllamaClient (the historical name preserved post-#1066 for call-site backward compatibility — a future rename to LlmClient is non-breaking and tracked separately).

Constants§

BACKEND_OLLAMA
v0.7.x (#1174 PR4-remainder) — canonical name of the Ollama-native backend selector.
OPENAI_COMPAT_EMBEDDINGS_PATH
#1598 — OpenAI-compatible embeddings endpoint path suffix, appended to the resolved base URL (e.g. https://openrouter.ai/api/v1). Single source for the embed wire path across the client’s two embed call sites and the doctor reachability probe.