# ic-rig
A lean, modular Rust library for building LLM agent applications — designed to run anywhere, including ICP (Internet Computer Protocol) WASM canisters.
**No HTTP client included.** You bring your own, which means `ic-rig` works on native Tokio, ICP canisters, WASM runtimes, or anywhere else you can make an HTTP call.
---
## Table of Contents
- [Features](#features)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Bringing Your Own HTTP Client](#bringing-your-own-http-client)
- [Providers](#providers)
- [OpenAI](#openai)
- [Anthropic](#anthropic)
- [Google Gemini](#google-gemini)
- [DeepSeek](#deepseek)
- [Building an Agent with Tools](#building-an-agent-with-tools)
- [Embeddings](#embeddings)
- [Vector Search with LSH](#vector-search-with-lsh)
- [ICP Canister Example](#icp-canister-example)
- [API Reference](#api-reference)
---
## Features
- **Provider-agnostic** — OpenAI, Anthropic, Google Gemini, and DeepSeek behind a single `CompletionModel` trait
- **Agentic tool-call loop** — built-in multi-turn reasoning with automatic tool dispatch
- **Embeddings + semantic search** — batch embeddings with LSH approximate nearest-neighbor indexing and configurable distance metrics
- **WASM/ICP-first** — no `Send + Sync` requirements on futures, no bundled HTTP dependencies
- **Minimal footprint** — only `serde`, `serde_json`, and `thiserror` are required
---
## Installation
Add `ic-rig` to your `Cargo.toml` and enable the providers you need:
```toml
[dependencies]
ic-rig = { version = "0.1", features = ["openai"] }
# or
ic-rig = { version = "0.1", features = ["anthropic"] }
# or
ic-rig = { version = "0.1", features = ["openai", "anthropic", "gemini", "deepseek"] }
```
Available feature flags:
| `openai` | OpenAI completion + embeddings |
| `anthropic` | Anthropic Claude completion |
| `gemini` | Google Gemini completion + embeddings |
| `deepseek` | DeepSeek completion |
---
## Quick Start
```rust
use ic_rig::Agent;
use ic_rig::providers::openai::{self, GPT_5_6};
// my_http implements ic_rig::http::HttpClient
let client = openai::Client::new(my_http, "sk-...");
let model = client.model(GPT_5_6);
let agent = Agent::builder(model)
.preamble("You are a concise, helpful assistant.")
.max_tokens(256)
.build();
let reply: String = agent.prompt("What is the capital of France?").await?;
println!("{reply}"); // "Paris."
```
---
## Bringing Your Own HTTP Client
`ic-rig` ships no HTTP client. You implement the `HttpClient` trait once for your platform, then pass it to any provider client.
```rust
use ic_rig::http::{HttpClient, HttpRequest, HttpResponse};
pub struct MyHttpClient;
impl HttpClient for MyHttpClient {
type Error = String;
async fn post(&self, req: HttpRequest) -> Result<HttpResponse, Self::Error> {
// req.url — target URL
// req.headers — Vec<(String, String)>
// req.body — Vec<u8> (always JSON)
todo!("implement for your runtime")
}
}
```
### Native (reqwest) example
```rust
use ic_rig::http::{HttpClient, HttpRequest, HttpResponse};
pub struct ReqwestClient(reqwest::Client);
impl HttpClient for ReqwestClient {
type Error = reqwest::Error;
async fn post(&self, req: HttpRequest) -> Result<HttpResponse, reqwest::Error> {
let mut builder = self.0.post(&req.url);
for (k, v) in req.headers {
builder = builder.header(k, v);
}
let resp = builder.body(req.body).send().await?;
let status = resp.status().as_u16();
let body = resp.bytes().await?.to_vec();
Ok(HttpResponse { status, body })
}
}
```
---
## Providers
### OpenAI
```rust
use ic_rig::providers::openai::{self, GPT_5_6, TEXT_EMBEDDING_3_SMALL};
let client = openai::Client::new(my_http, std::env::var("OPENAI_API_KEY").unwrap());
let model = client.model(GPT_5_6);
let embedder = client.embedding_model(TEXT_EMBEDDING_3_SMALL).with_dimensions(256);
```
Current generation (recommended): `GPT_5_6`, `GPT_5_6_TERRA`, `GPT_5_6_LUNA`, `GPT_5_6_CYBER`, `GPT_5_3_CODEX`
Previous generation (still active): `GPT_5`, `GPT_5_MINI`, `GPT_5_NANO`, `GPT_4_1`, `GPT_4_1_MINI`, `GPT_4_1_NANO`, `GPT_4O`, `GPT_4O_MINI`, `O3`, `O3_MINI`, `O4_MINI`
Available embedding models: `TEXT_EMBEDDING_3_LARGE`, `TEXT_EMBEDDING_3_SMALL`, `TEXT_EMBEDDING_ADA_002`
> **Note:** `GPT_4_TURBO` and `GPT_35_TURBO` are still exported but scheduled for removal by OpenAI on 2026-10-23 — migrate to `GPT_5_6`/`GPT_5_6_TERRA` and `GPT_5_6_TERRA`/`GPT_5_6_LUNA` respectively. `O1` and `O1_MINI` are exported but `#[deprecated]` — OpenAI already retired them (2025-07-28 and 2025-10-27); use `O3`/`O4_MINI`.
---
### Anthropic
```rust
use ic_rig::providers::anthropic::{self, CLAUDE_SONNET_5};
let client = anthropic::Client::new(my_http, std::env::var("ANTHROPIC_API_KEY").unwrap());
let model = client.model(CLAUDE_SONNET_5);
```
Current generation (recommended): `CLAUDE_FABLE_5`, `CLAUDE_OPUS_5`, `CLAUDE_SONNET_5`, `CLAUDE_HAIKU_4_5`
Previous generation (still active): `CLAUDE_OPUS_4_8`, `CLAUDE_OPUS_4_7`, `CLAUDE_OPUS_4_6`, `CLAUDE_SONNET_4_6`, `CLAUDE_OPUS_4_5`, `CLAUDE_SONNET_4_5`
> **Note:** `CLAUDE_OPUS_4` and `CLAUDE_SONNET_4` are still exported but `#[deprecated]` — Anthropic has deprecated them in favor of the 5-series (retirement date TBD). `CLAUDE_SONNET_4_5`'s snapshot date was corrected from `-20251101` to the real `-20250929` on 2026-08-27; update any code that hardcoded the old string instead of the constant.
>
> **Note:** Anthropic requires `max_tokens`. The default is 1024 if you don't call `.max_tokens()` on the builder.
---
### Google Gemini
```rust
use ic_rig::providers::gemini::{self, GEMINI_3_5_FLASH, GEMINI_EMBEDDING_001};
let client = gemini::Client::new(my_http, std::env::var("GEMINI_API_KEY").unwrap());
let model = client.model(GEMINI_3_5_FLASH);
let embedder = client.embedding_model(GEMINI_EMBEDDING_001);
```
Gemini 3 (current generation, recommended): `GEMINI_3_1_PRO_PREVIEW`, `GEMINI_3_7_FLASH`, `GEMINI_3_6_FLASH`, `GEMINI_3_5_FLASH`, `GEMINI_3_5_FLASH_LITE`, `GEMINI_3_1_FLASH_LITE`
Gemini 2.5 (previous generation; GA-stable until 2026-10-16): `GEMINI_2_5_PRO`, `GEMINI_2_5_FLASH`, `GEMINI_2_5_FLASH_LITE`
Available embedding models: `GEMINI_EMBEDDING_001`, `GEMINI_EMBEDDING_2_PREVIEW` (multimodal)
> **Note:** `GEMINI_2_0_FLASH`, `GEMINI_2_0_FLASH_LITE`, `GEMINI_1_5_PRO`, `GEMINI_1_5_FLASH`, `TEXT_EMBEDDING_004`, and `EMBEDDING_001` are still exported but `#[deprecated]` — Google has already shut all of them down. `GEMINI_2_5_PRO` was repointed from the dated preview ID `gemini-2.5-pro-preview-05-06` (which Google now redirects) to the stable `gemini-2.5-pro`.
---
### DeepSeek
DeepSeek's API is OpenAI-compatible. In thinking mode the model produces a chain-of-thought `reasoning_content` field; when `content` is empty ic-rig surfaces the reasoning trace as the reply so the agent loop always gets a usable string.
```rust
use ic_rig::providers::deepseek::{self, DEEPSEEK_V4_FLASH, DEEPSEEK_V4_PRO};
let client = deepseek::Client::new(my_http, std::env::var("DEEPSEEK_API_KEY").unwrap());
let flash = client.model(DEEPSEEK_V4_FLASH);
let pro = client.model(DEEPSEEK_V4_PRO);
```
Available models: `DEEPSEEK_V4_FLASH`, `DEEPSEEK_V4_PRO`, `DEEPSEEK_V4_FLASH_VISION_EXP` (experimental, multimodal)
> **Note:** DeepSeek does not provide an embeddings API; only completion is supported.
>
> `DEEPSEEK_CHAT` and `DEEPSEEK_REASONER` are still exported but `#[deprecated]` — DeepSeek fully retired those model IDs on 2026-07-24. Switch to `DEEPSEEK_V4_FLASH` (their replacement in both non-thinking and thinking mode).
---
## Building an Agent with Tools
`ic-rig` has a built-in agentic loop that automatically dispatches tool calls and feeds results back to the model until it returns a final text response.
### 1. Define a tool
```rust
use ic_rig::tool::{Tool, ToolDefinition};
use serde::{Deserialize, Serialize};
use serde_json::json;
#[derive(Deserialize)]
struct WeatherArgs { city: String }
#[derive(Serialize)]
struct WeatherResult { temperature_c: f32, condition: String }
pub struct WeatherTool;
impl Tool for WeatherTool {
const NAME: &'static str = "get_weather";
type Error = String;
type Args = WeatherArgs;
type Output = WeatherResult;
fn definition(&self) -> ToolDefinition {
ToolDefinition {
name: Self::NAME.into(),
description: "Get the current weather for a city.".into(),
parameters: json!({
"type": "object",
"properties": {
"city": { "type": "string", "description": "City name, e.g. 'London'" }
},
"required": ["city"]
}),
}
}
async fn call(&self, args: WeatherArgs) -> Result<WeatherResult, String> {
Ok(WeatherResult { temperature_c: 18.5, condition: format!("Sunny in {}", args.city) })
}
}
```
### 2. Register tools and run the agent
```rust
let agent = Agent::builder(model)
.preamble("You are a helpful weather assistant.")
.tool(WeatherTool)
.max_tokens(512)
.max_iterations(5)
.build();
let reply = agent.prompt("What is the weather in Tokyo right now?").await?;
// "The current weather in Tokyo is 18.5°C and sunny."
```
### 3. Multi-tool agents
```rust
let agent = Agent::builder(model)
.preamble("You are a research assistant.")
.tool(WeatherTool)
.tool(NewsTool)
.tool(CalculatorTool)
.max_tokens(1024)
.build();
```
### 4. Injecting context (RAG)
```rust
let agent = Agent::builder(model)
.preamble("Answer using only the provided context.")
.context(retrieved_chunk_1)
.context(retrieved_chunk_2)
.build();
```
### 5. Multi-turn chat
`prompt()` is single-turn. For a back-and-forth conversation, use `chat()` and maintain a `Vec<Message>` history yourself:
```rust
use ic_rig::completion::Message;
let agent = Agent::builder(model).preamble("You are helpful.").build();
let mut history: Vec<Message> = Vec::new();
let r1 = agent.chat("My name is Alice.", &mut history).await?;
let r2 = agent.chat("What is my name?", &mut history).await?;
// r2 => "Your name is Alice."
```
### 6. Thinking / reasoning models
`agent.prompt()` and `agent.chat()` always return the model's final answer — a chain-of-thought / "thinking" trace, if the provider produces one, never ends up mixed into that string. Some models (DeepSeek's reasoning models, Claude's extended thinking, Gemini's thinking models) think by default; use `.thinking(false)` if you just want a straight answer, or `.thinking(true)` to make sure it's on:
```rust
let agent = Agent::builder(model)
.preamble("You are a helpful assistant.")
.thinking(false) // straight answer, no reasoning trace
.build();
let reply = agent.prompt("What is 17 * 24?").await?;
// reply is just "408" — no "<thinking>..." trace mixed in, even on a
// reasoning model that would otherwise produce one.
```
Leaving `.thinking(...)` unset keeps the provider's own default behavior. Each provider maps this to its own request parameter (DeepSeek's `thinking.type`, Anthropic's `thinking.type`, Gemini's `thinkingConfig`, OpenAI's `reasoning_effort`) — see [`CompletionRequest::thinking`](src/completion.rs) for the exact per-provider translation and its caveats (e.g. Gemini 3.1 Pro can't fully disable thinking, and pre-4.6 Claude snapshots don't support this at all).
If you're calling a `CompletionModel` directly instead of going through `Agent`, the reasoning trace (when present) is on `CompletionResponse::reasoning` — a separate field from `choice`, so you can show or log it without it ever contaminating the answer:
```rust
let response = model.complete(request).await?;
if let Some(trace) = &response.reasoning {
println!("(thinking: {trace})");
}
```
---
## Embeddings
### Embedding a list of strings
```rust
use ic_rig::embeddings::EmbeddingsBuilder;
use ic_rig::providers::openai::{self, TEXT_EMBEDDING_3_SMALL};
let model = openai::Client::new(my_http, api_key).embedding_model(TEXT_EMBEDDING_3_SMALL);
let results = EmbeddingsBuilder::new(model)
.document("The Eiffel Tower is in Paris.".to_string())?
.document("Mount Fuji is in Japan.".to_string())?
.build()
.await?;
for (text, embeddings) in results {
println!("{}: {:?}", text, &embeddings[0].vec[..4]);
}
```
### Embedding custom types
Implement the `Embed` trait to embed multiple fields per document:
```rust
use ic_rig::embeddings::{Embed, TextEmbedder, EmbedError};
struct Article { id: u64, title: String, body: String }
impl Embed for Article {
fn embed(&self, e: &mut TextEmbedder) -> Result<(), EmbedError> {
e.embed(self.title.clone());
e.embed(self.body.clone());
Ok(())
}
}
let results = EmbeddingsBuilder::new(model)
.documents(articles)? // Vec<Article>
.build()
.await?;
// results: Vec<(Article, Vec<Embedding>)>
// Each article gets two Embeddings: one for title, one for body.
```
### Distance metrics
Every `Embedding` implements `VectorDistance`, giving you direct access to all metrics:
```rust
use ic_rig::embeddings::VectorDistance;
let sim = a.cosine_similarity(&b, false); // false = vectors not pre-normalised
let dist = a.euclidean_distance(&b);
let dot = a.dot_product(&b);
```
When the metric is a runtime value (config, user choice), use `DistanceMetric` instead:
```rust
use ic_rig::DistanceMetric;
let metric = DistanceMetric::Cosine { normalized: false };
let score = metric.score(&query_embedding, &candidate_embedding);
```
Available variants:
| `Cosine { normalized: bool }` | `[-1, 1]` | descending |
| `Angular { normalized: bool }` | `[0, 1]` | ascending |
| `Euclidean` | `[0, ∞)` | ascending |
| `Manhattan` | `[0, ∞)` | ascending |
| `Chebyshev` | `[0, ∞)` | ascending |
| `DotProduct` | `(-∞, ∞)` | descending |
---
## Vector Search with LSH
`LshIndex` provides fast approximate nearest-neighbor search using locality-sensitive hashing. It is designed for use inside ICP canisters where you cannot run an external vector database.
### Indexing
```rust
use ic_rig::vector_store::lsh::LshIndex;
// new(dimensions, num_hyperplanes, num_tables, seed)
let mut index = LshIndex::new(1536, 12, 6, 42);
index.insert("doc-1".into(), &embedding_1.vec);
index.insert("doc-2".into(), &embedding_2.vec);
index.insert("doc-3".into(), &embedding_3.vec);
```
### Searching
`search()` runs the LSH lookup, scores the candidates, and returns sorted `(id, score)` pairs in one call. Pass `None` for the metric to default to cosine similarity:
```rust
use ic_rig::DistanceMetric;
use std::collections::HashMap;
// store maps each ID to its Embedding
let results: Vec<(String, f64)> = index.search(
&query_embedding,
&store,
None, // default: Cosine { normalized: false }
);
// or pick a metric explicitly
let results = index.search(&query_embedding, &store, Some(DistanceMetric::Euclidean));
// results are already sorted best-first
for (id, score) in &results {
println!("{id}: {score:.4}");
}
```
### Low-level query
If you need the raw candidate IDs without scoring, use `query()` directly:
```rust
let candidates: Vec<String> = index.query(&query_vec);
// score and sort candidates yourself
```
### Tuning
LSH trades recall for speed. The two parameters control the tradeoff:
| `num_hyperplanes` | Fewer candidates, faster scoring | More candidates, slower |
| `num_tables` | Better recall, more RAM | Less RAM, more misses |
A good starting point for 1536-dimensional OpenAI embeddings: `LshIndex::new(1536, 12, 6, 42)`.
---
## ICP Canister Example
```rust
use ic_cdk::update;
use ic_rig::Agent;
use ic_rig::providers::anthropic::{self, CLAUDE_HAIKU_4_5};
mod http {
use ic_rig::http::{HttpClient, HttpRequest, HttpResponse};
pub struct IcpClient;
impl HttpClient for IcpClient {
type Error = String;
async fn post(&self, req: HttpRequest) -> Result<HttpResponse, String> {
use ic_cdk::api::management_canister::http_request::{
http_request, CanisterHttpRequestArgument, HttpMethod,
};
let args = CanisterHttpRequestArgument {
url: req.url,
method: HttpMethod::POST,
headers: req.headers.into_iter()
.map(|(k, v)| ic_cdk::api::management_canister::http_request::HttpHeader {
name: k, value: v,
})
.collect(),
body: Some(req.body),
..Default::default()
};
let (resp,) = http_request(args, 50_000_000_000).await.map_err(|(_, e)| e)?;
Ok(HttpResponse { status: resp.status.0.try_into().unwrap_or(500), body: resp.body })
}
}
}
#[update]
async fn ask(question: String) -> String {
let api_key = /* load from stable storage */;
let client = anthropic::Client::new(http::IcpClient, api_key);
let agent = Agent::builder(client.model(CLAUDE_HAIKU_4_5))
.preamble("You are a helpful assistant running on the Internet Computer.")
.max_tokens(512)
.build();
agent.prompt(question).await.unwrap_or_else(|e| format!("Error: {e}"))
}
```
---
## API Reference
### `Agent<M>`
| `Agent::builder(model)` | Create an `AgentBuilder` |
| `.prompt(text)` | Single-turn prompt, returns `String` |
| `.chat(text, history)` | Multi-turn prompt, updates history in place |
### `AgentBuilder<M>`
| `.preamble(s)` | Set the system prompt |
| `.tool(t)` | Register a tool |
| `.temperature(f)` | Sampling temperature |
| `.max_tokens(n)` | Maximum output tokens |
| `.thinking(bool)` | Explicitly turn thinking/reasoning mode on or off (default: provider's own default) |
| `.max_iterations(n)` | Maximum tool-call rounds (default: 10) |
| `.context(s)` | Inject a RAG chunk into the conversation |
| `.build()` | Produce `Agent<M>` |
### `EmbeddingsBuilder<M, T>`
| `EmbeddingsBuilder::new(model)` | Create a builder |
| `.document(d)` | Add a single document |
| `.documents(ds)` | Add many documents |
| `.build()` | Embed all documents, returns `Vec<(T, Vec<Embedding>)>` |
### `DistanceMetric`
| `.score(a, b)` | Score two embeddings using this metric |
| `.higher_is_better()` | `true` for similarity metrics, `false` for distance metrics |
### `LshIndex`
| `LshIndex::new(dim, hyperplanes, tables, seed)` | Create an index |
| `.insert(id, vec)` | Index a vector under a string ID |
| `.search(query, store, metric)` | LSH lookup + score + sort; `metric` is `Option<DistanceMetric>` |
| `.query(vec)` | Raw LSH lookup — returns unscored candidate IDs |
| `.len()` / `.is_empty()` | Size queries |
| `.clear()` | Remove all entries |
---
## License
MIT