af-llm 0.2.0

Unified async LLM client with retry, timeout and circuit breaking. Talks to any OpenAI-compatible endpoint (LiteLLM proxy, DeepSeek, Anthropic-via-proxy, ...).
Documentation

af-llm — unified async LLM access for the Agent Factory platform.

Provider-neutral model transport and request types. Provides:

  • [LlmClient] — async chat completions against any OpenAI-compatible endpoint, with a hard per-request timeout and a shared [CircuitBreaker].
  • Strongly-typed request/response models ([CompletionRequest], [CompletionResponse], [ChatMessage], [Tool], …) — the Pydantic equivalent, validated at the wire boundary.
  • [parse_json] — pull structured output out of fenced LLM text into any serde type.

Example

use af_llm::{LlmClient, LlmConfig, CompletionRequest, ChatMessage};

# async fn run() -> af_llm::Result<()> {
let client = LlmClient::new(LlmConfig::new("http://localhost:4000/v1", ""))?;
let req = CompletionRequest::new(
    "deepseek/deepseek-chat",
    vec![ChatMessage::user("Say hi in one word.")],
);
let resp = client.complete_stream_single_attempt(&req, |_, _| {}).await?;
println!("{:?}", resp.first_content());
# Ok(())
# }