# Daimon Provider Guide
This document covers every LLM and embedding provider in the Daimon framework, with configuration details, feature flags, and practical usage patterns.
---
## Provider Overview
| OpenAI | `openai` (default) | ✓ | ✓ | ✓ | ✓ (system) | `OpenAiCostModel` | — |
| Anthropic | `anthropic` (default) | ✓ | ✓ | — | ✓ (cache_control) | `AnthropicCostModel` | — |
| Ollama | `ollama` (default) | ✓ | ✓ | ✓ | — | — | — |
| Google Gemini | `gemini` | ✓ | ✓ | ✓ | ✓ (cached_content) | — | `PubSubBroker` (pubsub) |
| Azure OpenAI | `azure` | ✓ | ✓ | ✓ | ✓ (system) | — | `ServiceBusBroker` (servicebus) |
| AWS Bedrock | `bedrock` | ✓ | ✓ | ✓ | ✓ (Claude) | — | `SqsBroker` (sqs) |
---
## OpenAI (default)
**Feature:** `openai` (included in default features)
### Chat Model
```rust
use daimon::model::openai::OpenAi;
use daimon::prelude::*;
use std::time::Duration;
// From environment (OPENAI_API_KEY)
let model = OpenAi::new("gpt-4o");
// With explicit API key
let model = OpenAi::with_api_key("gpt-4o", std::env::var("OPENAI_API_KEY")?)
.with_base_url("https://api.openai.com/v1") // or proxy/local endpoint
.with_timeout(Duration::from_secs(60))
.with_max_retries(5)
.with_response_format("json_object") // JSON mode
.with_parallel_tool_calls(true);
```
**Configuration methods:**
| `.with_api_key(model_id, key)` | Explicit API key (otherwise reads `OPENAI_API_KEY`) |
| `.with_base_url(url)` | Custom base URL (proxies, local endpoints) |
| `.with_timeout(duration)` | HTTP request timeout |
| `.with_max_retries(n)` | Retries for 429 and 5xx (default: 3) |
| `.with_response_format(format)` | `"json_object"` or `"text"` |
| `.with_parallel_tool_calls(enabled)` | Allow multiple tool calls per turn |
**Models:** `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`, `gpt-4`, `gpt-3.5-turbo`, `o1`, `o3-mini`
**Capabilities:** Tool calls, streaming, JSON mode, parallel tool calls, response format. Caching: system message caching via API (reported in `usage.cached_tokens`).
### Embedding
```rust
use daimon::model::openai_embed::OpenAiEmbedding;
let embedding = OpenAiEmbedding::new("text-embedding-3-small")
.with_api_key(api_key)
.with_base_url("https://api.openai.com/v1")
.with_dimensions(1536); // 1536 for small, 3072 for large
```
**Models:** `text-embedding-3-small` (1536 dims), `text-embedding-3-large` (3072 dims)
### Cost Model
```rust
use daimon::cost::OpenAiCostModel;
let agent = Agent::builder()
.model(OpenAi::new("gpt-4o"))
.cost_model(OpenAiCostModel)
.max_budget(0.50)
.build()?;
```
---
## Anthropic (default)
**Feature:** `anthropic` (included in default features)
### Chat Model
```rust
use daimon::model::anthropic::Anthropic;
use std::time::Duration;
let model = Anthropic::new("claude-sonnet-4-20250514");
let model = Anthropic::with_api_key("claude-sonnet-4-20250514", std::env::var("ANTHROPIC_API_KEY")?)
.with_base_url("https://api.anthropic.com")
.with_timeout(Duration::from_secs(60))
.with_max_retries(5)
.with_prompt_caching(); // cache_control breakpoints for system + tools
```
**Configuration methods:**
| `.with_api_key(model_id, key)` | Explicit API key (otherwise `ANTHROPIC_API_KEY`) |
| `.with_base_url(url)` | Custom base URL |
| `.with_timeout(duration)` | HTTP timeout |
| `.with_max_retries(n)` | Retries for 429, 529, 5xx (default: 3) |
| `.with_prompt_caching()` | Enables `cache_control` breakpoints (system, tools) |
**Models:** `claude-sonnet-4-20250514`, `claude-3-5-haiku`, `claude-3-opus`, `claude-3-haiku`, etc.
**Capabilities:** Tool calls, streaming, overloaded retry (429/529/5xx). Caching: native `cache_control` breakpoints for system and tool definitions; `usage.cached_tokens` reports cache reads.
### Cost Model
```rust
use daimon::cost::AnthropicCostModel;
let agent = Agent::builder()
.model(Anthropic::new("claude-sonnet-4-20250514"))
.cost_model(AnthropicCostModel)
.build()?;
```
---
## Ollama (default)
**Feature:** `ollama` (included in default features)
### Chat Model
```rust
use daimon::model::ollama::Ollama;
use std::time::Duration;
let model = Ollama::new("llama3.2");
let model = Ollama::new("llama3.2")
.with_base_url("http://localhost:11434") // default
.with_timeout(Duration::from_secs(300))
.with_keep_alive("5m"); // keep model loaded; "0" to unload immediately
```
**Configuration methods:**
| `.with_base_url(url)` | Ollama server URL (default: `http://localhost:11434`) |
| `.with_timeout(duration)` | Request timeout (default: 300s) |
| `.with_keep_alive(duration_str)` | e.g. `"5m"`, `"1h"`, `"0"` to unload |
**Models:** Any model served by Ollama (e.g. `llama3.2`, `llama3.1`, `mistral`, `codellama`). Tool calls are model-dependent.
**Capabilities:** Tool calls (if model supports), streaming. No caching. No cost model (local/free).
### Embedding
```rust
use daimon::model::ollama_embed::OllamaEmbedding;
let embedding = OllamaEmbedding::new("nomic-embed-text")
.with_base_url("http://localhost:11434")
.with_dimensions(768); // model-dependent; nomic-embed-text is 768
```
Uses `OLLAMA_HOST` env var if set; otherwise `http://localhost:11434`.
---
## Google Gemini (feature = "gemini")
**Feature:** `gemini` — enables `daimon-provider-gemini`
### Chat Model
```rust
use daimon::model::gemini::Gemini;
use std::time::Duration;
// From GOOGLE_API_KEY
let model = Gemini::new("gemini-2.0-flash");
let model = Gemini::with_api_key("gemini-2.0-flash", std::env::var("GOOGLE_API_KEY")?)
.with_base_url("https://generativelanguage.googleapis.com/v1beta") // or Vertex AI URL
.with_timeout(Duration::from_secs(60))
.with_max_retries(5)
.with_bearer_token() // for Vertex AI (OAuth2)
.with_cached_content("cachedContents/<id>"); // pre-cached system/tools
```
**Configuration methods:**
| `.with_api_key(model_id, key)` | Explicit key (otherwise `GOOGLE_API_KEY`) |
| `.with_base_url(url)` | Custom URL (e.g. Vertex AI) |
| `.with_timeout(duration)` | HTTP timeout |
| `.with_max_retries(n)` | Retries for 429, 5xx |
| `.with_bearer_token()` | Use `Authorization: Bearer` (Vertex AI) |
| `.with_cached_content(name)` | Reference pre-created cached content |
**Models:** `gemini-2.0-flash`, `gemini-pro`, `gemini-1.5-pro`, `gemini-1.5-flash`
**Capabilities:** Tool calls, streaming. Caching: system instruction caching via `with_cached_content` or Gemini Caching API.
### Embedding
```rust
use daimon::model::gemini::GeminiEmbedding;
let embedding = GeminiEmbedding::new("text-embedding-004")
.with_api_key(api_key)
.with_base_url("https://generativelanguage.googleapis.com/v1beta")
.with_dimensions(768)
.with_bearer_token();
```
### Cloud Broker
With feature `pubsub`: `PubSubBroker` for Google Cloud Pub/Sub task distribution.
---
## Azure OpenAI (feature = "azure")
**Feature:** `azure` — enables `daimon-provider-azure`
### Chat Model
```rust
use daimon::model::azure::AzureOpenAi;
use std::time::Duration;
// From AZURE_OPENAI_API_KEY
let model = AzureOpenAi::new(
"https://my-resource.openai.azure.com",
"gpt-4o", // deployment name
);
let model = AzureOpenAi::with_api_key(
"https://my-resource.openai.azure.com",
"gpt-4o",
std::env::var("AZURE_OPENAI_API_KEY")?,
)
.with_api_version("2024-10-21")
.with_timeout(Duration::from_secs(60))
.with_max_retries(5)
.with_bearer_token(); // for Microsoft Entra ID
```
**Configuration methods:**
| `.with_api_key(resource_url, deployment, key)` | Explicit key (otherwise `AZURE_OPENAI_API_KEY`) |
| `.with_api_version(version)` | API version (default: `2024-10-21`) |
| `.with_timeout(duration)` | HTTP timeout |
| `.with_max_retries(n)` | Retries |
| `.with_bearer_token()` | Microsoft Entra ID (Azure AD) auth |
**Endpoint format:** `{resource_url}/openai/deployments/{deployment}/chat/completions?api-version=...`
**Capabilities:** Same as OpenAI — tool calls, streaming, JSON mode. Caching: system message caching (same as OpenAI).
### Embedding
```rust
use daimon::model::azure::AzureOpenAiEmbedding;
let embedding = AzureOpenAiEmbedding::new(
"https://my-resource.openai.azure.com",
"text-embedding-3-small",
)
.with_api_key(key)
.with_api_version("2024-10-21")
.with_dimensions(1536)
.with_bearer_token();
```
### Cloud Broker
With feature `servicebus`: `ServiceBusBroker` for Azure Service Bus task distribution.
---
## AWS Bedrock (feature = "bedrock")
**Feature:** `bedrock` — enables `daimon-provider-bedrock`
### Chat Model
```rust
use daimon::model::bedrock::Bedrock;
use std::time::Duration;
let model = Bedrock::new("us.anthropic.claude-sonnet-4-20250514-v1:0")
.with_region("us-east-1")
.with_max_retries(5)
.with_guardrail("guardrail-id", "DRAFT")
.with_prompt_caching(); // CachePoint for system + tools (Claude)
```
**Configuration methods:**
| `.with_client(client)` | Use a pre-built Bedrock client |
| `.with_region(region)` | AWS region (otherwise from env/config) |
| `.with_max_retries(n)` | Retries for throttling/5xx |
| `.with_guardrail(id, version)` | Content filtering guardrail |
| `.with_prompt_caching()` | CachePoint blocks for system and tools (Claude models) |
**Authentication:** Uses AWS SDK default credential chain (env vars, `~/.aws/credentials`, IAM roles).
**Models:** Anthropic Claude (`us.anthropic.claude-*`), Amazon Titan, Meta Llama, Cohere, AI21 — use full Bedrock model IDs.
**Capabilities:** Tool calls, streaming, guardrails. Caching: native system/tool caching for Claude models via `with_prompt_caching()`.
### Embedding
```rust
use daimon::model::bedrock::BedrockEmbedding;
let embedding = BedrockEmbedding::new("amazon.titan-embed-text-v2:0")
.with_region("us-east-1")
.with_dimensions(1024)
.with_normalize(true);
```
### Cloud Broker
With feature `sqs`: `SqsBroker` for AWS SQS task distribution.
---
## Switching Providers at Runtime
### SharedModel and Arc<dyn ErasedModel>
All providers implement the `Model` trait. Use `Arc<dyn ErasedModel>` (aliased as `SharedModel`) for dynamic dispatch when the provider is chosen at runtime:
```rust
use daimon::model::SharedModel;
use daimon::model::openai::OpenAi;
use daimon::model::anthropic::Anthropic;
use std::sync::Arc;
fn select_model(use_openai: bool) -> SharedModel {
if use_openai {
Arc::new(OpenAi::new("gpt-4o"))
} else {
Arc::new(Anthropic::new("claude-sonnet-4-20250514"))
}
}
let agent = Agent::builder()
.shared_model(select_model(std::env::var("USE_OPENAI").is_ok()))
.build()?;
```
### HotSwapAgent for Runtime Model Swapping
`HotSwapAgent` wraps an agent behind a `RwLock`, allowing you to swap the model (or tools, system prompt, memory) at runtime without rebuilding:
```rust
use daimon::prelude::*;
use daimon::agent::hot_swap::HotSwapAgent;
use daimon::model::openai::OpenAi;
use daimon::model::anthropic::Anthropic;
let agent = Agent::builder()
.model(OpenAi::new("gpt-4o"))
.system_prompt("You are helpful.")
.build()?;
let hot = HotSwapAgent::new(agent);
// Use normally
let response = hot.prompt("Hello").await?;
// Swap model at runtime
hot.swap_model(Anthropic::new("claude-sonnet-4-20250514")).await;
// Next prompt uses the new model
let response = hot.prompt("Hello again").await?;
// Or swap with a pre-boxed SharedModel
hot.swap_shared_model(Arc::new(OpenAi::new("gpt-4o-mini"))).await;
```
### A/B Testing with fork_builder
Use `Agent::fork_builder()` to create mutated copies for A/B testing or specialized variants:
```rust
let base = Agent::builder()
.model(OpenAi::new("gpt-4o"))
.system_prompt("You are a helpful assistant.")
.tool(SearchTool)
.build()?;
// Variant A: different model
let variant_a = base.fork_builder()
.model(Anthropic::new("claude-sonnet-4-20250514"))
.build();
// Variant B: different system prompt
let variant_b = base.fork_builder()
.system_prompt("You are a code reviewer. Be strict.")
.remove_tool("search")
.tool(ReviewTool)
.build();
// Run both and compare
let resp_a = variant_a.prompt("Review this code").await?;
let resp_b = variant_b.prompt("Review this code").await?;
```
---
## Provider-Specific Caching
| **OpenAI** | System message caching | Automatic via API; `usage.cached_tokens` reports reads |
| **Anthropic** | `cache_control` breakpoints | `.with_prompt_caching()` — system + tools |
| **Bedrock** | CachePoint blocks (Claude) | `.with_prompt_caching()` — system + tools |
| **Gemini** | System instruction / Caching API | `.with_cached_content("cachedContents/<id>")` or create via API |
| **Azure OpenAI** | Same as OpenAI | Automatic; `usage.cached_tokens` |
| **Ollama** | — | No native caching |
---
## Environment Variables
All providers support reading API keys from the environment. Set these before running:
| OpenAI | `OPENAI_API_KEY` |
| Anthropic | `ANTHROPIC_API_KEY` |
| Ollama | `OLLAMA_HOST` (optional; default `http://localhost:11434`) |
| Gemini | `GOOGLE_API_KEY` |
| Azure OpenAI | `AZURE_OPENAI_API_KEY` |
| Bedrock | AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`) or IAM role |
Example:
```rust
// Typical pattern: read from env, fallback for tests
let api_key = std::env::var("OPENAI_API_KEY")
.unwrap_or_else(|_| "sk-test-placeholder".to_string());
let model = OpenAi::with_api_key("gpt-4o", api_key);
```
---
## Cargo.toml Feature Selection
```toml
[dependencies]
# Default: openai, anthropic, ollama, macros
daimon = "0.16"
# Minimal: only OpenAI
daimon = { version = "0.16", default-features = false, features = ["openai"] }
# Add Gemini and Azure
daimon = { version = "0.16", features = ["gemini", "azure"] }
# Full: all providers + MCP, SQLite, Redis, etc.
daimon = { version = "0.16", features = ["full"] }
```
| `openai` | OpenAI chat + embedding |
| `anthropic` | Anthropic Claude |
| `ollama` | Ollama chat + embedding |
| `gemini` | Gemini chat + embedding |
| `azure` | Azure OpenAI chat + embedding |
| `bedrock` | Bedrock chat + embedding |
| `sqs` | Bedrock + SqsBroker |
| `pubsub` | Gemini + PubSubBroker |
| `servicebus` | Azure + ServiceBusBroker |