Skip to main content

Module client

drep::llm

Module client 

Source
Expand description

The LLM client.

One boundary: LlmClient::complete_json takes a system prompt and a user payload, sends them to the configured OpenAI-compatible endpoint, and returns the JSON value the model produced. Cache and concurrency limiting live above this single-provider request boundary.

§What is deliberately delegated to the SDK

  • Streaming. open-agent-sdk parses the SSE stream; drep concatenates the ContentBlock::Text blocks it emits and ignores the rest. Since 0.10.0 those blocks are fragments - one event per delta, delivered while the stream is open, where 0.9.x emitted the whole response as a single block at the end. The types are identical either way, so nothing here failed to compile and nothing here changed: the join in run_one_query is what makes the assembled text independent of where the deltas fall. Reading one block as the whole answer would now return a prefix, and src/llm/client/tests/streaming.rs is what would notice.
  • Transport retry. retry_with_backoff_conditional decides per error whether to retry (5xx, timeout, stream error) or fail fast (4xx, config errors). drep adds no retry layer on top.

§What this module owns

  • Parse retry. The same prompt truncates the same way, so a parse failure does NOT retry. The retry closure returns Ok(None) for an unparseable body; the SDK’s retry sees Ok(...) and stops.
  • Attempt count floor. LlmConfig::max_retries may be 0, but a “zero attempts loop” would skip the request and report a bogus “no exception was captured”. The floor is 1.
  • max_tokens pass-through. The configured cap is forwarded only when the user set one. open-agent-sdk 0.7.0 omits the field entirely otherwise, so “unset” means the server decides - which is what a 256k-1M context model needs. (Before 0.7.0 the builder substituted 4096 and truncated reasoning models mid-thought; drep passed a large sentinel to work around it. That workaround is gone.)

Structs§

LlmClient
A configured LLM client ready to issue requests.

Constants§

NO_JSON_ATTEMPTS
How many times a response carrying no JSON at all is asked for again.