llm-universal 0.1.0

# llm-universal

One async Rust client for **OpenAI-compatible Chat Completions** and the **Anthropic Messages API**.
Write your chat and tool-calling code once and switch providers by switching the constructor.

- `AnthropicProvider` — `POST /v1/messages`
- `OpenAiProvider` — `POST /chat/completions`; also works with any OpenAI-compatible server
  (Ollama, vLLM, LM Studio, OpenRouter, Groq) via `with_base_url`
- Tool calling with one normalized format for both providers
- Errors modeled on the official SDKs (`BadRequest`, `Authentication`, `RateLimit`, …, request IDs, retry rules)
- Client-side validation before any network call
- Safe defaults: connect/request timeouts, no redirects (the key never goes to another host),
  API keys redacted from `Debug`

## Install

```toml
[dependencies]
llm-universal = "0.1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
```

## Quick start

```rust,no_run
use llm_universal::{AnthropicProvider, ChatRequest, LlmProvider, Message, OpenAiProvider};

#[tokio::main]
async fn main() -> llm_universal::Result<()> {
    let provider: Box<dyn LlmProvider> = match std::env::var("ANTHROPIC_API_KEY") {
        Ok(key) => Box::new(AnthropicProvider::new(key)),
        Err(_) => Box::new(OpenAiProvider::new(std::env::var("OPENAI_API_KEY").unwrap_or_default())),
    };

    let req = ChatRequest::new("claude-opus-5-5")
        .system("Be concise.")
        .max_tokens(512)
        .message(Message::user("Hello!"));

    let resp = provider.chat(&req).await?;
    println!("{}", resp.text());
    Ok(())
}
```

Local OpenAI-compatible server:

```rust,no_run
# use llm_universal::OpenAiProvider;
let ollama = OpenAiProvider::new("").with_base_url("http://localhost:11434/v1");
```

## Tool calling

```rust,no_run
# use llm_universal::*;
# use serde_json::json;
# async fn run(provider: &dyn LlmProvider) -> Result<()> {
let weather = Tool::new(
    "get_weather",
    "Get the current weather in a city",
    json!({ "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] }),
);
let mut req = ChatRequest::new("claude-opus-5-5")
    .tool(weather)
    .message(Message::user("What's the weather in Berlin?"));

let resp = provider.chat(&req).await?;
if resp.stop_reason == StopReason::ToolUse {
    let calls = resp.tool_calls();
    req.messages.push(resp.message);
    let results = calls
        .into_iter()
        .map(|(id, _name, _args)| Part::ToolResult { call_id: id, content: "18°C, cloudy".into(), is_error: false })
        .collect();
    req.messages.push(Message::new(Role::Tool, results));
    let final_resp = provider.chat(&req).await?;
    println!("{}", final_resp.text());
}
# Ok(()) }
```

A complete loop is in [`examples/tools.rs`](examples/tools.rs): `cargo run --example tools`.

## Errors

| `LlmError` variant | When | SDK equivalent |
|---|---|---|
| `Api(Box<ApiError>)` | non-2xx response | `APIStatusError` and subclasses |
| `Connection` | DNS, TLS, dropped connection | `APIConnectionError` |
| `Timeout` | request timed out | `APIConnectionTimeoutError` |
| `Deserialize` | body is not JSON | `JSONDeserialize` (async-openai) |
| `InvalidResponse` | JSON is missing expected fields | — |
| `InvalidArgument` | client-side validation failed | `InvalidArgument` (async-openai) |

`ApiError` carries `status`, `kind` (`ApiErrorKind::RateLimit`, …), `message`, `error_type`,
`code`, `param`, `request_id` and the JSON `body`. Its message format matches the SDKs:

```text
401 invalid x-api-key
429 Rate limit reached for requests
invalid args: model must not be empty
```

```rust,no_run
# use llm_universal::*;
# async fn run(provider: &dyn LlmProvider, req: &ChatRequest) {
match provider.chat(req).await {
    Ok(resp) => println!("{}", resp.text()),
    Err(err) if err.is_retryable() => eprintln!("retry later: {err} (request id: {:?})", err.request_id()),
    Err(err) => eprintln!("{err}"),
}
# }
```

`is_retryable()` follows the SDK retry rules: 408, 409, 429, >= 500, connection errors and timeouts.

## Notes

- Anthropic newer models accept only `temperature = 1.0`; leave `temperature` unset unless you need it.
- Extended-thinking blocks are not part of the normalized message; if you enable thinking, keep the raw
  response to send signed thinking blocks back unchanged.
- Streaming is not supported yet.

## License

Licensed under either of [Apache License, Version 2.0](LICENSE-APACHE) or [MIT license](LICENSE-MIT) at your option.