Skip to main content

Module llm

Module llm 

Source
Expand description

LLM client and JSON extraction.

Split by concern:

  • json_parsing turns whatever the model returned into a serde_json::Value. Tolerant of fences, prose, trailing commas and truncated output, in that order, first success wins.
  • client wraps open-agent-sdk. The request itself - streaming, transport retry, and parse retry.
  • cache is the on-disk response cache. Content-addressed keys (not git-aware), infallible reads, oldest-first eviction.
  • concurrency is the bounded in-flight limiter. Deliberately just a Semaphore; see the module doc for why it remains narrow.
  • models asks an endpoint which models it serves, for drep init.
  • quirks answers what one model accepts - temperature and its output ceiling - which no endpoint’s listing carries.
  • chain is the ordered list of providers with failover. It owns the loop, so the cache key is recomputed per provider and the answer is filed under the key of whoever gave it.

chain::ProviderChain::complete_json is the boundary the analyzer calls; client::LlmClient::complete_json is the single-provider request beneath it. It returns the Extracted from json_parsing, never the raw text, so a truncated response stays a type the caller can pattern-match on rather than a log line to grep for.

Modules§

backend
Backend-specific request machinery behind the provider-chain contract.
cache
Content-addressed cache for LLM responses.
chain
The provider chain: an ordered list of LLM providers, tried in turn.
client
The LLM client.
codex
ChatGPT-subscription reviews through the separately installed Codex CLI.
concurrency
Concurrency cap for LLM requests.
error
Backend-neutral failures at the LLM boundary.
json_parsing
Tolerant JSON extraction from LLM responses.
models
Asking an endpoint which models it serves.
quirks
What one model accepts, as opposed to what its provider usually does.