af-llm — unified async LLM access for the Agent Factory platform.
Provider-neutral model transport and request types. Provides:
- [
LlmClient] — async chat completions against any OpenAI-compatible endpoint, with a hard per-request timeout and a shared [CircuitBreaker]. - Strongly-typed request/response models ([
CompletionRequest], [CompletionResponse], [ChatMessage], [Tool], …) — the Pydantic equivalent, validated at the wire boundary. - [
parse_json] — pull structured output out of fenced LLM text into anyserdetype.
Example
use af_llm::{LlmClient, LlmConfig, CompletionRequest, ChatMessage};
# async fn run() -> af_llm::Result<()> {
let client = LlmClient::new(LlmConfig::new("http://localhost:4000/v1", ""))?;
let req = CompletionRequest::new(
"deepseek/deepseek-chat",
vec![ChatMessage::user("Say hi in one word.")],
);
let resp = client.complete_stream_single_attempt(&req, |_, _| {}).await?;
println!("{:?}", resp.first_content());
# Ok(())
# }