ic-rig

A lean, modular Rust library for building LLM agent applications — designed to run anywhere, including ICP (Internet Computer Protocol) WASM canisters.

No HTTP client included. You bring your own, which means ic-rig works on native Tokio, ICP canisters, WASM runtimes, or anywhere else you can make an HTTP call.

Features
Installation
Quick Start
Bringing Your Own HTTP Client
Providers
Building an Agent with Tools
Embeddings
Vector Search with LSH
ICP Canister Example
API Reference

Features

Provider-agnostic — OpenAI, Anthropic, Google Gemini, and DeepSeek behind a single CompletionModel trait
Agentic tool-call loop — built-in multi-turn reasoning with automatic tool dispatch
Embeddings + semantic search — batch embeddings with LSH approximate nearest-neighbor indexing and configurable distance metrics
WASM/ICP-first — no Send + Sync requirements on futures, no bundled HTTP dependencies
Minimal footprint — only serde, serde_json, and thiserror are required

Installation

Add ic-rig to your Cargo.toml and enable the providers you need:

[dependencies]
ic-rig = { version = "0.1", features = ["openai"] }
# or
ic-rig = { version = "0.1", features = ["anthropic"] }
# or
ic-rig = { version = "0.1", features = ["openai", "anthropic", "gemini", "deepseek"] }

Available feature flags:

Flag	Enables
`openai`	OpenAI completion + embeddings
`anthropic`	Anthropic Claude completion
`gemini`	Google Gemini completion + embeddings
`deepseek`	DeepSeek completion

Quick Start

use ic_rig::Agent;
use ic_rig::providers::openai::{self, GPT_5_6};

// my_http implements ic_rig::http::HttpClient
let client = openai::Client::new(my_http, "sk-...");
let model  = client.model(GPT_5_6);

let agent = Agent::builder(model)
    .preamble("You are a concise, helpful assistant.")
    .max_tokens(256)
    .build();

let reply: String = agent.prompt("What is the capital of France?").await?;
println!("{reply}"); // "Paris."

Bringing Your Own HTTP Client

ic-rig ships no HTTP client. You implement the HttpClient trait once for your platform, then pass it to any provider client.

use ic_rig::http::{HttpClient, HttpRequest, HttpResponse};

pub struct MyHttpClient;

impl HttpClient for MyHttpClient {
    type Error = String;

    async fn post(&self, req: HttpRequest) -> Result<HttpResponse, Self::Error> {
        // req.url      — target URL
        // req.headers  — Vec<(String, String)>
        // req.body     — Vec<u8> (always JSON)
        todo!("implement for your runtime")
    }
}

Native (reqwest) example

use ic_rig::http::{HttpClient, HttpRequest, HttpResponse};

pub struct ReqwestClient(reqwest::Client);

impl HttpClient for ReqwestClient {
    type Error = reqwest::Error;

    async fn post(&self, req: HttpRequest) -> Result<HttpResponse, reqwest::Error> {
        let mut builder = self.0.post(&req.url);
        for (k, v) in req.headers {
            builder = builder.header(k, v);
        }
        let resp = builder.body(req.body).send().await?;
        let status = resp.status().as_u16();
        let body   = resp.bytes().await?.to_vec();
        Ok(HttpResponse { status, body })
    }
}

Providers

OpenAI

use ic_rig::providers::openai::{self, GPT_5_6, TEXT_EMBEDDING_3_SMALL};

let client = openai::Client::new(my_http, std::env::var("OPENAI_API_KEY").unwrap());

let model    = client.model(GPT_5_6);
let embedder = client.embedding_model(TEXT_EMBEDDING_3_SMALL).with_dimensions(256);

Current generation (recommended): GPT_5_6, GPT_5_6_TERRA, GPT_5_6_LUNA, GPT_5_6_CYBER, GPT_5_3_CODEX

Previous generation (still active): GPT_5, GPT_5_MINI, GPT_5_NANO, GPT_4_1, GPT_4_1_MINI, GPT_4_1_NANO, GPT_4O, GPT_4O_MINI, O3, O3_MINI, O4_MINI

Available embedding models: TEXT_EMBEDDING_3_LARGE, TEXT_EMBEDDING_3_SMALL, TEXT_EMBEDDING_ADA_002

Note: GPT_4_TURBO and GPT_35_TURBO are still exported but scheduled for removal by OpenAI on 2026-10-23 — migrate to GPT_5_6/GPT_5_6_TERRA and GPT_5_6_TERRA/GPT_5_6_LUNA respectively. O1 and O1_MINI are exported but #[deprecated] — OpenAI already retired them (2025-07-28 and 2025-10-27); use O3/O4_MINI.

Anthropic

use ic_rig::providers::anthropic::{self, CLAUDE_SONNET_5};

let client = anthropic::Client::new(my_http, std::env::var("ANTHROPIC_API_KEY").unwrap());
let model  = client.model(CLAUDE_SONNET_5);

Current generation (recommended): CLAUDE_FABLE_5, CLAUDE_OPUS_5, CLAUDE_SONNET_5, CLAUDE_HAIKU_4_5

Previous generation (still active): CLAUDE_OPUS_4_8, CLAUDE_OPUS_4_7, CLAUDE_OPUS_4_6, CLAUDE_SONNET_4_6, CLAUDE_OPUS_4_5, CLAUDE_SONNET_4_5

Note: CLAUDE_OPUS_4 and CLAUDE_SONNET_4 are still exported but #[deprecated] — Anthropic has deprecated them in favor of the 5-series (retirement date TBD). CLAUDE_SONNET_4_5's snapshot date was corrected from -20251101 to the real -20250929 on 2026-08-27; update any code that hardcoded the old string instead of the constant.

Note: Anthropic requires max_tokens. The default is 1024 if you don't call .max_tokens() on the builder.

Google Gemini

use ic_rig::providers::gemini::{self, GEMINI_3_5_FLASH, GEMINI_EMBEDDING_001};

let client   = gemini::Client::new(my_http, std::env::var("GEMINI_API_KEY").unwrap());
let model    = client.model(GEMINI_3_5_FLASH);
let embedder = client.embedding_model(GEMINI_EMBEDDING_001);

Gemini 3 (current generation, recommended): GEMINI_3_1_PRO_PREVIEW, GEMINI_3_7_FLASH, GEMINI_3_6_FLASH, GEMINI_3_5_FLASH, GEMINI_3_5_FLASH_LITE, GEMINI_3_1_FLASH_LITE

Gemini 2.5 (previous generation; GA-stable until 2026-10-16): GEMINI_2_5_PRO, GEMINI_2_5_FLASH, GEMINI_2_5_FLASH_LITE

Available embedding models: GEMINI_EMBEDDING_001, GEMINI_EMBEDDING_2_PREVIEW (multimodal)

Note: GEMINI_2_0_FLASH, GEMINI_2_0_FLASH_LITE, GEMINI_1_5_PRO, GEMINI_1_5_FLASH, TEXT_EMBEDDING_004, and EMBEDDING_001 are still exported but #[deprecated] — Google has already shut all of them down. GEMINI_2_5_PRO was repointed from the dated preview ID gemini-2.5-pro-preview-05-06 (which Google now redirects) to the stable gemini-2.5-pro.

DeepSeek

DeepSeek's API is OpenAI-compatible. In thinking mode the model produces a chain-of-thought reasoning_content field; when content is empty ic-rig surfaces the reasoning trace as the reply so the agent loop always gets a usable string.

use ic_rig::providers::deepseek::{self, DEEPSEEK_V4_FLASH, DEEPSEEK_V4_PRO};

let client = deepseek::Client::new(my_http, std::env::var("DEEPSEEK_API_KEY").unwrap());
let flash  = client.model(DEEPSEEK_V4_FLASH);
let pro    = client.model(DEEPSEEK_V4_PRO);

Available models: DEEPSEEK_V4_FLASH, DEEPSEEK_V4_PRO, DEEPSEEK_V4_FLASH_VISION_EXP (experimental, multimodal)

Note: DeepSeek does not provide an embeddings API; only completion is supported.

DEEPSEEK_CHAT and DEEPSEEK_REASONER are still exported but #[deprecated] — DeepSeek fully retired those model IDs on 2026-07-24. Switch to DEEPSEEK_V4_FLASH (their replacement in both non-thinking and thinking mode).

Building an Agent with Tools

ic-rig has a built-in agentic loop that automatically dispatches tool calls and feeds results back to the model until it returns a final text response.

1. Define a tool

use ic_rig::tool::{Tool, ToolDefinition};
use serde::{Deserialize, Serialize};
use serde_json::json;

#[derive(Deserialize)]
struct WeatherArgs { city: String }

#[derive(Serialize)]
struct WeatherResult { temperature_c: f32, condition: String }

pub struct WeatherTool;

impl Tool for WeatherTool {
    const NAME: &'static str = "get_weather";

    type Error  = String;
    type Args   = WeatherArgs;
    type Output = WeatherResult;

    fn definition(&self) -> ToolDefinition {
        ToolDefinition {
            name: Self::NAME.into(),
            description: "Get the current weather for a city.".into(),
            parameters: json!({
                "type": "object",
                "properties": {
                    "city": { "type": "string", "description": "City name, e.g. 'London'" }
                },
                "required": ["city"]
            }),
        }
    }

    async fn call(&self, args: WeatherArgs) -> Result<WeatherResult, String> {
        Ok(WeatherResult { temperature_c: 18.5, condition: format!("Sunny in {}", args.city) })
    }
}

2. Register tools and run the agent

let agent = Agent::builder(model)
    .preamble("You are a helpful weather assistant.")
    .tool(WeatherTool)
    .max_tokens(512)
    .max_iterations(5)
    .build();

let reply = agent.prompt("What is the weather in Tokyo right now?").await?;
// "The current weather in Tokyo is 18.5°C and sunny."

3. Multi-tool agents

let agent = Agent::builder(model)
    .preamble("You are a research assistant.")
    .tool(WeatherTool)
    .tool(NewsTool)
    .tool(CalculatorTool)
    .max_tokens(1024)
    .build();

4. Injecting context (RAG)

let agent = Agent::builder(model)
    .preamble("Answer using only the provided context.")
    .context(retrieved_chunk_1)
    .context(retrieved_chunk_2)
    .build();

5. Multi-turn chat

prompt() is single-turn. For a back-and-forth conversation, use chat() and maintain a Vec<Message> history yourself:

use ic_rig::completion::Message;

let agent = Agent::builder(model).preamble("You are helpful.").build();
let mut history: Vec<Message> = Vec::new();

let r1 = agent.chat("My name is Alice.", &mut history).await?;
let r2 = agent.chat("What is my name?", &mut history).await?;
// r2 => "Your name is Alice."

6. Thinking / reasoning models

agent.prompt() and agent.chat() always return the model's final answer — a chain-of-thought / "thinking" trace, if the provider produces one, never ends up mixed into that string. Some models (DeepSeek's reasoning models, Claude's extended thinking, Gemini's thinking models) think by default; use .thinking(false) if you just want a straight answer, or .thinking(true) to make sure it's on:

let agent = Agent::builder(model)
    .preamble("You are a helpful assistant.")
    .thinking(false) // straight answer, no reasoning trace
    .build();

let reply = agent.prompt("What is 17 * 24?").await?;
// reply is just "408" — no "<thinking>..." trace mixed in, even on a
// reasoning model that would otherwise produce one.

Leaving .thinking(...) unset keeps the provider's own default behavior. Each provider maps this to its own request parameter (DeepSeek's thinking.type, Anthropic's thinking.type, Gemini's thinkingConfig, OpenAI's reasoning_effort) — see CompletionRequest::thinking for the exact per-provider translation and its caveats (e.g. Gemini 3.1 Pro can't fully disable thinking, and pre-4.6 Claude snapshots don't support this at all).

If you're calling a CompletionModel directly instead of going through Agent, the reasoning trace (when present) is on CompletionResponse::reasoning — a separate field from choice, so you can show or log it without it ever contaminating the answer:

let response = model.complete(request).await?;
if let Some(trace) = &response.reasoning {
    println!("(thinking: {trace})");
}

Embeddings

Embedding a list of strings

use ic_rig::embeddings::EmbeddingsBuilder;
use ic_rig::providers::openai::{self, TEXT_EMBEDDING_3_SMALL};

let model   = openai::Client::new(my_http, api_key).embedding_model(TEXT_EMBEDDING_3_SMALL);
let results = EmbeddingsBuilder::new(model)
    .document("The Eiffel Tower is in Paris.".to_string())?
    .document("Mount Fuji is in Japan.".to_string())?
    .build()
    .await?;

for (text, embeddings) in results {
    println!("{}: {:?}", text, &embeddings[0].vec[..4]);
}

Embedding custom types

Implement the Embed trait to embed multiple fields per document:

use ic_rig::embeddings::{Embed, TextEmbedder, EmbedError};

struct Article { id: u64, title: String, body: String }

impl Embed for Article {
    fn embed(&self, e: &mut TextEmbedder) -> Result<(), EmbedError> {
        e.embed(self.title.clone());
        e.embed(self.body.clone());
        Ok(())
    }
}

let results = EmbeddingsBuilder::new(model)
    .documents(articles)?  // Vec<Article>
    .build()
    .await?;

// results: Vec<(Article, Vec<Embedding>)>
// Each article gets two Embeddings: one for title, one for body.

Distance metrics

Every Embedding implements VectorDistance, giving you direct access to all metrics:

use ic_rig::embeddings::VectorDistance;

let sim  = a.cosine_similarity(&b, false); // false = vectors not pre-normalised
let dist = a.euclidean_distance(&b);
let dot  = a.dot_product(&b);

When the metric is a runtime value (config, user choice), use DistanceMetric instead:

use ic_rig::DistanceMetric;

let metric = DistanceMetric::Cosine { normalized: false };
let score  = metric.score(&query_embedding, &candidate_embedding);

Available variants:

Variant	Range	Best-first sort
`Cosine { normalized: bool }`	`[-1, 1]`	descending
`Angular { normalized: bool }`	`[0, 1]`	ascending
`Euclidean`	`[0, ∞)`	ascending
`Manhattan`	`[0, ∞)`	ascending
`Chebyshev`	`[0, ∞)`	ascending
`DotProduct`	`(-∞, ∞)`	descending

Vector Search with LSH

LshIndex provides fast approximate nearest-neighbor search using locality-sensitive hashing. It is designed for use inside ICP canisters where you cannot run an external vector database.

Indexing

use ic_rig::vector_store::lsh::LshIndex;

// new(dimensions, num_hyperplanes, num_tables, seed)
let mut index = LshIndex::new(1536, 12, 6, 42);

index.insert("doc-1".into(), &embedding_1.vec);
index.insert("doc-2".into(), &embedding_2.vec);
index.insert("doc-3".into(), &embedding_3.vec);

Searching

search() runs the LSH lookup, scores the candidates, and returns sorted (id, score) pairs in one call. Pass None for the metric to default to cosine similarity:

use ic_rig::DistanceMetric;
use std::collections::HashMap;

// store maps each ID to its Embedding
let results: Vec<(String, f64)> = index.search(
    &query_embedding,
    &store,
    None,                                 // default: Cosine { normalized: false }
);

// or pick a metric explicitly
let results = index.search(&query_embedding, &store, Some(DistanceMetric::Euclidean));

// results are already sorted best-first
for (id, score) in &results {
    println!("{id}: {score:.4}");
}

Low-level query

If you need the raw candidate IDs without scoring, use query() directly:

let candidates: Vec<String> = index.query(&query_vec);
// score and sort candidates yourself

Tuning

LSH trades recall for speed. The two parameters control the tradeoff:

Parameter	Higher value	Lower value
`num_hyperplanes`	Fewer candidates, faster scoring	More candidates, slower
`num_tables`	Better recall, more RAM	Less RAM, more misses

A good starting point for 1536-dimensional OpenAI embeddings: LshIndex::new(1536, 12, 6, 42).

ICP Canister Example

use ic_cdk::update;
use ic_rig::Agent;
use ic_rig::providers::anthropic::{self, CLAUDE_HAIKU_4_5};

mod http {
    use ic_rig::http::{HttpClient, HttpRequest, HttpResponse};

    pub struct IcpClient;

    impl HttpClient for IcpClient {
        type Error = String;

        async fn post(&self, req: HttpRequest) -> Result<HttpResponse, String> {
            use ic_cdk::api::management_canister::http_request::{
                http_request, CanisterHttpRequestArgument, HttpMethod,
            };
            let args = CanisterHttpRequestArgument {
                url: req.url,
                method: HttpMethod::POST,
                headers: req.headers.into_iter()
                    .map(|(k, v)| ic_cdk::api::management_canister::http_request::HttpHeader {
                        name: k, value: v,
                    })
                    .collect(),
                body: Some(req.body),
                ..Default::default()
            };
            let (resp,) = http_request(args, 50_000_000_000).await.map_err(|(_, e)| e)?;
            Ok(HttpResponse { status: resp.status.0.try_into().unwrap_or(500), body: resp.body })
        }
    }
}

#[update]
async fn ask(question: String) -> String {
    let api_key = /* load from stable storage */;
    let client  = anthropic::Client::new(http::IcpClient, api_key);
    let agent   = Agent::builder(client.model(CLAUDE_HAIKU_4_5))
        .preamble("You are a helpful assistant running on the Internet Computer.")
        .max_tokens(512)
        .build();

    agent.prompt(question).await.unwrap_or_else(|e| format!("Error: {e}"))
}

API Reference

`Agent<M>`

Method	Description
`Agent::builder(model)`	Create an `AgentBuilder`
`.prompt(text)`	Single-turn prompt, returns `String`
`.chat(text, history)`	Multi-turn prompt, updates history in place

`AgentBuilder<M>`

Method	Description
`.preamble(s)`	Set the system prompt
`.tool(t)`	Register a tool
`.temperature(f)`	Sampling temperature
`.max_tokens(n)`	Maximum output tokens
`.thinking(bool)`	Explicitly turn thinking/reasoning mode on or off (default: provider's own default)
`.max_iterations(n)`	Maximum tool-call rounds (default: 10)
`.context(s)`	Inject a RAG chunk into the conversation
`.build()`	Produce `Agent<M>`

`EmbeddingsBuilder<M, T>`

Method	Description
`EmbeddingsBuilder::new(model)`	Create a builder
`.document(d)`	Add a single document
`.documents(ds)`	Add many documents
`.build()`	Embed all documents, returns `Vec<(T, Vec<Embedding>)>`

`DistanceMetric`

Method	Description
`.score(a, b)`	Score two embeddings using this metric
`.higher_is_better()`	`true` for similarity metrics, `false` for distance metrics

`LshIndex`

Method	Description
`LshIndex::new(dim, hyperplanes, tables, seed)`	Create an index
`.insert(id, vec)`	Index a vector under a string ID
`.search(query, store, metric)`	LSH lookup + score + sort; `metric` is `Option<DistanceMetric>`
`.query(vec)`	Raw LSH lookup — returns unscored candidate IDs
`.len()` / `.is_empty()`	Size queries
`.clear()`	Remove all entries

License

MIT

ic-rig 0.2.0