llmkit-rs 0.1.0

Unified multi-provider async LLM client for Rust — OpenAI, Anthropic, Ollama, with Tower middleware
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

# llmkit

Unified multi-provider async LLM client for Rust — OpenAI, Anthropic, and local
Ollama behind one `LlmProvider` trait, one streaming API, and a Tower middleware
stack (retry, rate limiting, cost tracking, tracing, fallback).

```rust
use llmkit::prelude::*;

#[tokio::main]
async fn main() -> LlmResult<()> {
    let client = LlmClientBuilder::new()
        .provider(AnthropicProvider::from_env()?.model("claude-opus-4-8"))
        .fallback(OpenAiProvider::from_env()?.model("gpt-4o-mini"))
        .layer(RetryLayer::exponential(3, std::time::Duration::from_millis(200)))
        .build()?;

    let resp = client.chat(ChatRequest::builder().user("Hello!").build()).await?;
    println!("{}", resp.text().unwrap_or_default());
    Ok(())
}
```

See the [repository](https://github.com/codiux-net/llmkit-rs) for the full guide,
examples, and the per-provider crates.

## Cargo features

- `openai`, `anthropic`, `ollama` — provider adapters (all enabled by default)

## License

Apache-2.0 OR MIT.