ic-rig 0.1.0

A lean, modular library for building LLM applications. Bring your own HTTP client.
Documentation

ic-rig

A lean, modular Rust library for building LLM agent applications — designed to run anywhere, including ICP (Internet Computer Protocol) WASM canisters.

No HTTP client included. You bring your own, which means ic-rig works on native Tokio, ICP canisters, WASM runtimes, or anywhere else you can make an HTTP call.

Table of Contents

Features

  • Provider-agnostic — OpenAI, Anthropic, Google Gemini, and DeepSeek behind a single CompletionModel trait
  • Agentic tool-call loop — built-in multi-turn reasoning with automatic tool dispatch
  • Embeddings + semantic search — batch embeddings with LSH approximate nearest-neighbor indexing and configurable distance metrics
  • WASM/ICP-first — no Send + Sync requirements on futures, no bundled HTTP dependencies
  • Minimal footprint — only serde, serde_json, and thiserror are required

Installation

Add ic-rig to your Cargo.toml and enable the providers you need:

[dependencies]
ic-rig = { version = "0.1", features = ["openai"] }
# or
ic-rig = { version = "0.1", features = ["anthropic"] }
# or
ic-rig = { version = "0.1", features = ["openai", "anthropic", "gemini", "deepseek"] }

Available feature flags:

Flag Enables
openai OpenAI completion + embeddings
anthropic Anthropic Claude completion
gemini Google Gemini completion + embeddings
deepseek DeepSeek completion

Quick Start

use ic_rig::Agent;
use ic_rig::providers::openai::{self, GPT_5_6};

// my_http implements ic_rig::http::HttpClient
let client = openai::Client::new(my_http, "sk-...");
let model  = client.model(GPT_5_6);

let agent = Agent::builder(model)
    .preamble("You are a concise, helpful assistant.")
    .max_tokens(256)
    .build();

let reply: String = agent.prompt("What is the capital of France?").await?;
println!("{reply}"); // "Paris."

Bringing Your Own HTTP Client

ic-rig ships no HTTP client. You implement the HttpClient trait once for your platform, then pass it to any provider client.

use ic_rig::http::{HttpClient, HttpRequest, HttpResponse};

pub struct MyHttpClient;

impl HttpClient for MyHttpClient {
    type Error = String;

    async fn post(&self, req: HttpRequest) -> Result<HttpResponse, Self::Error> {
        // req.url      — target URL
        // req.headers  — Vec<(String, String)>
        // req.body     — Vec<u8> (always JSON)
        todo!("implement for your runtime")
    }
}

Native (reqwest) example

use ic_rig::http::{HttpClient, HttpRequest, HttpResponse};

pub struct ReqwestClient(reqwest::Client);

impl HttpClient for ReqwestClient {
    type Error = reqwest::Error;

    async fn post(&self, req: HttpRequest) -> Result<HttpResponse, reqwest::Error> {
        let mut builder = self.0.post(&req.url);
        for (k, v) in req.headers {
            builder = builder.header(k, v);
        }
        let resp = builder.body(req.body).send().await?;
        let status = resp.status().as_u16();
        let body   = resp.bytes().await?.to_vec();
        Ok(HttpResponse { status, body })
    }
}

Providers

OpenAI

use ic_rig::providers::openai::{self, GPT_5_6, TEXT_EMBEDDING_3_SMALL};

let client = openai::Client::new(my_http, std::env::var("OPENAI_API_KEY").unwrap());

let model    = client.model(GPT_5_6);
let embedder = client.embedding_model(TEXT_EMBEDDING_3_SMALL).with_dimensions(256);

Current generation (recommended): GPT_5_6, GPT_5_6_TERRA, GPT_5_6_LUNA, GPT_5_6_CYBER, GPT_5_3_CODEX

Previous generation (still active): GPT_5, GPT_5_MINI, GPT_5_NANO, GPT_4_1, GPT_4_1_MINI, GPT_4_1_NANO, GPT_4O, GPT_4O_MINI, O3, O3_MINI, O4_MINI

Available embedding models: TEXT_EMBEDDING_3_LARGE, TEXT_EMBEDDING_3_SMALL, TEXT_EMBEDDING_ADA_002

Note: GPT_4_TURBO and GPT_35_TURBO are still exported but scheduled for removal by OpenAI on 2026-10-23 — migrate to GPT_5_6/GPT_5_6_TERRA and GPT_5_6_TERRA/GPT_5_6_LUNA respectively. O1 and O1_MINI are exported but #[deprecated] — OpenAI already retired them (2025-07-28 and 2025-10-27); use O3/O4_MINI.

Anthropic

use ic_rig::providers::anthropic::{self, CLAUDE_SONNET_5};

let client = anthropic::Client::new(my_http, std::env::var("ANTHROPIC_API_KEY").unwrap());
let model  = client.model(CLAUDE_SONNET_5);

Current generation (recommended): CLAUDE_FABLE_5, CLAUDE_OPUS_5, CLAUDE_SONNET_5, CLAUDE_HAIKU_4_5

Previous generation (still active): CLAUDE_OPUS_4_8, CLAUDE_OPUS_4_7, CLAUDE_OPUS_4_6, CLAUDE_SONNET_4_6, CLAUDE_OPUS_4_5, CLAUDE_SONNET_4_5

Note: CLAUDE_OPUS_4 and CLAUDE_SONNET_4 are still exported but #[deprecated] — Anthropic has deprecated them in favor of the 5-series (retirement date TBD). CLAUDE_SONNET_4_5's snapshot date was corrected from -20251101 to the real -20250929 on 2026-08-27; update any code that hardcoded the old string instead of the constant.

Note: Anthropic requires max_tokens. The default is 1024 if you don't call .max_tokens() on the builder.

Google Gemini

use ic_rig::providers::gemini::{self, GEMINI_3_5_FLASH, GEMINI_EMBEDDING_001};

let client   = gemini::Client::new(my_http, std::env::var("GEMINI_API_KEY").unwrap());
let model    = client.model(GEMINI_3_5_FLASH);
let embedder = client.embedding_model(GEMINI_EMBEDDING_001);

Gemini 3 (current generation, recommended): GEMINI_3_1_PRO_PREVIEW, GEMINI_3_7_FLASH, GEMINI_3_6_FLASH, GEMINI_3_5_FLASH, GEMINI_3_5_FLASH_LITE, GEMINI_3_1_FLASH_LITE

Gemini 2.5 (previous generation; GA-stable until 2026-10-16): GEMINI_2_5_PRO, GEMINI_2_5_FLASH, GEMINI_2_5_FLASH_LITE

Available embedding models: GEMINI_EMBEDDING_001, GEMINI_EMBEDDING_2_PREVIEW (multimodal)

Note: GEMINI_2_0_FLASH, GEMINI_2_0_FLASH_LITE, GEMINI_1_5_PRO, GEMINI_1_5_FLASH, TEXT_EMBEDDING_004, and EMBEDDING_001 are still exported but #[deprecated] — Google has already shut all of them down. GEMINI_2_5_PRO was repointed from the dated preview ID gemini-2.5-pro-preview-05-06 (which Google now redirects) to the stable gemini-2.5-pro.

DeepSeek

DeepSeek's API is OpenAI-compatible. In thinking mode the model produces a chain-of-thought reasoning_content field; when content is empty ic-rig surfaces the reasoning trace as the reply so the agent loop always gets a usable string.

use ic_rig::providers::deepseek::{self, DEEPSEEK_V4_FLASH, DEEPSEEK_V4_PRO};

let client = deepseek::Client::new(my_http, std::env::var("DEEPSEEK_API_KEY").unwrap());
let flash  = client.model(DEEPSEEK_V4_FLASH);
let pro    = client.model(DEEPSEEK_V4_PRO);

Available models: DEEPSEEK_V4_FLASH, DEEPSEEK_V4_PRO, DEEPSEEK_V4_FLASH_VISION_EXP (experimental, multimodal)

Note: DeepSeek does not provide an embeddings API; only completion is supported.

DEEPSEEK_CHAT and DEEPSEEK_REASONER are still exported but #[deprecated] — DeepSeek fully retired those model IDs on 2026-07-24. Switch to DEEPSEEK_V4_FLASH (their replacement in both non-thinking and thinking mode).

Building an Agent with Tools

ic-rig has a built-in agentic loop that automatically dispatches tool calls and feeds results back to the model until it returns a final text response.

1. Define a tool

use ic_rig::tool::{Tool, ToolDefinition};
use serde::{Deserialize, Serialize};
use serde_json::json;

#[derive(Deserialize)]
struct WeatherArgs { city: String }

#[derive(Serialize)]
struct WeatherResult { temperature_c: f32, condition: String }

pub struct WeatherTool;

impl Tool for WeatherTool {
    const NAME: &'static str = "get_weather";

    type Error  = String;
    type Args   = WeatherArgs;
    type Output = WeatherResult;

    fn definition(&self) -> ToolDefinition {
        ToolDefinition {
            name: Self::NAME.into(),
            description: "Get the current weather for a city.".into(),
            parameters: json!({
                "type": "object",
                "properties": {
                    "city": { "type": "string", "description": "City name, e.g. 'London'" }
                },
                "required": ["city"]
            }),
        }
    }

    async fn call(&self, args: WeatherArgs) -> Result<WeatherResult, String> {
        Ok(WeatherResult { temperature_c: 18.5, condition: format!("Sunny in {}", args.city) })
    }
}

2. Register tools and run the agent

let agent = Agent::builder(model)
    .preamble("You are a helpful weather assistant.")
    .tool(WeatherTool)
    .max_tokens(512)
    .max_iterations(5)
    .build();

let reply = agent.prompt("What is the weather in Tokyo right now?").await?;
// "The current weather in Tokyo is 18.5°C and sunny."

3. Multi-tool agents

let agent = Agent::builder(model)
    .preamble("You are a research assistant.")
    .tool(WeatherTool)
    .tool(NewsTool)
    .tool(CalculatorTool)
    .max_tokens(1024)
    .build();

4. Injecting context (RAG)

let agent = Agent::builder(model)
    .preamble("Answer using only the provided context.")
    .context(retrieved_chunk_1)
    .context(retrieved_chunk_2)
    .build();

5. Multi-turn chat

prompt() is single-turn. For a back-and-forth conversation, use chat() and maintain a Vec<Message> history yourself:

use ic_rig::completion::Message;

let agent = Agent::builder(model).preamble("You are helpful.").build();
let mut history: Vec<Message> = Vec::new();

let r1 = agent.chat("My name is Alice.", &mut history).await?;
let r2 = agent.chat("What is my name?", &mut history).await?;
// r2 => "Your name is Alice."

Embeddings

Embedding a list of strings

use ic_rig::embeddings::EmbeddingsBuilder;
use ic_rig::providers::openai::{self, TEXT_EMBEDDING_3_SMALL};

let model   = openai::Client::new(my_http, api_key).embedding_model(TEXT_EMBEDDING_3_SMALL);
let results = EmbeddingsBuilder::new(model)
    .document("The Eiffel Tower is in Paris.".to_string())?
    .document("Mount Fuji is in Japan.".to_string())?
    .build()
    .await?;

for (text, embeddings) in results {
    println!("{}: {:?}", text, &embeddings[0].vec[..4]);
}

Embedding custom types

Implement the Embed trait to embed multiple fields per document:

use ic_rig::embeddings::{Embed, TextEmbedder, EmbedError};

struct Article { id: u64, title: String, body: String }

impl Embed for Article {
    fn embed(&self, e: &mut TextEmbedder) -> Result<(), EmbedError> {
        e.embed(self.title.clone());
        e.embed(self.body.clone());
        Ok(())
    }
}

let results = EmbeddingsBuilder::new(model)
    .documents(articles)?  // Vec<Article>
    .build()
    .await?;

// results: Vec<(Article, Vec<Embedding>)>
// Each article gets two Embeddings: one for title, one for body.

Distance metrics

Every Embedding implements VectorDistance, giving you direct access to all metrics:

use ic_rig::embeddings::VectorDistance;

let sim  = a.cosine_similarity(&b, false); // false = vectors not pre-normalised
let dist = a.euclidean_distance(&b);
let dot  = a.dot_product(&b);

When the metric is a runtime value (config, user choice), use DistanceMetric instead:

use ic_rig::DistanceMetric;

let metric = DistanceMetric::Cosine { normalized: false };
let score  = metric.score(&query_embedding, &candidate_embedding);

Available variants:

Variant Range Best-first sort
Cosine { normalized: bool } [-1, 1] descending
Angular { normalized: bool } [0, 1] ascending
Euclidean [0, ∞) ascending
Manhattan [0, ∞) ascending
Chebyshev [0, ∞) ascending
DotProduct (-∞, ∞) descending

Vector Search with LSH

LshIndex provides fast approximate nearest-neighbor search using locality-sensitive hashing. It is designed for use inside ICP canisters where you cannot run an external vector database.

Indexing

use ic_rig::vector_store::lsh::LshIndex;

// new(dimensions, num_hyperplanes, num_tables, seed)
let mut index = LshIndex::new(1536, 12, 6, 42);

index.insert("doc-1".into(), &embedding_1.vec);
index.insert("doc-2".into(), &embedding_2.vec);
index.insert("doc-3".into(), &embedding_3.vec);

Searching

search() runs the LSH lookup, scores the candidates, and returns sorted (id, score) pairs in one call. Pass None for the metric to default to cosine similarity:

use ic_rig::DistanceMetric;
use std::collections::HashMap;

// store maps each ID to its Embedding
let results: Vec<(String, f64)> = index.search(
    &query_embedding,
    &store,
    None,                                 // default: Cosine { normalized: false }
);

// or pick a metric explicitly
let results = index.search(&query_embedding, &store, Some(DistanceMetric::Euclidean));

// results are already sorted best-first
for (id, score) in &results {
    println!("{id}: {score:.4}");
}

Low-level query

If you need the raw candidate IDs without scoring, use query() directly:

let candidates: Vec<String> = index.query(&query_vec);
// score and sort candidates yourself

Tuning

LSH trades recall for speed. The two parameters control the tradeoff:

Parameter Higher value Lower value
num_hyperplanes Fewer candidates, faster scoring More candidates, slower
num_tables Better recall, more RAM Less RAM, more misses

A good starting point for 1536-dimensional OpenAI embeddings: LshIndex::new(1536, 12, 6, 42).

ICP Canister Example

use ic_cdk::update;
use ic_rig::Agent;
use ic_rig::providers::anthropic::{self, CLAUDE_HAIKU_4_5};

mod http {
    use ic_rig::http::{HttpClient, HttpRequest, HttpResponse};

    pub struct IcpClient;

    impl HttpClient for IcpClient {
        type Error = String;

        async fn post(&self, req: HttpRequest) -> Result<HttpResponse, String> {
            use ic_cdk::api::management_canister::http_request::{
                http_request, CanisterHttpRequestArgument, HttpMethod,
            };
            let args = CanisterHttpRequestArgument {
                url: req.url,
                method: HttpMethod::POST,
                headers: req.headers.into_iter()
                    .map(|(k, v)| ic_cdk::api::management_canister::http_request::HttpHeader {
                        name: k, value: v,
                    })
                    .collect(),
                body: Some(req.body),
                ..Default::default()
            };
            let (resp,) = http_request(args, 50_000_000_000).await.map_err(|(_, e)| e)?;
            Ok(HttpResponse { status: resp.status.0.try_into().unwrap_or(500), body: resp.body })
        }
    }
}

#[update]
async fn ask(question: String) -> String {
    let api_key = /* load from stable storage */;
    let client  = anthropic::Client::new(http::IcpClient, api_key);
    let agent   = Agent::builder(client.model(CLAUDE_HAIKU_4_5))
        .preamble("You are a helpful assistant running on the Internet Computer.")
        .max_tokens(512)
        .build();

    agent.prompt(question).await.unwrap_or_else(|e| format!("Error: {e}"))
}

API Reference

Agent<M>

Method Description
Agent::builder(model) Create an AgentBuilder
.prompt(text) Single-turn prompt, returns String
.chat(text, history) Multi-turn prompt, updates history in place

AgentBuilder<M>

Method Description
.preamble(s) Set the system prompt
.tool(t) Register a tool
.temperature(f) Sampling temperature
.max_tokens(n) Maximum output tokens
.max_iterations(n) Maximum tool-call rounds (default: 10)
.context(s) Inject a RAG chunk into the conversation
.build() Produce Agent<M>

EmbeddingsBuilder<M, T>

Method Description
EmbeddingsBuilder::new(model) Create a builder
.document(d) Add a single document
.documents(ds) Add many documents
.build() Embed all documents, returns Vec<(T, Vec<Embedding>)>

DistanceMetric

Method Description
.score(a, b) Score two embeddings using this metric
.higher_is_better() true for similarity metrics, false for distance metrics

LshIndex

Method Description
LshIndex::new(dim, hyperplanes, tables, seed) Create an index
.insert(id, vec) Index a vector under a string ID
.search(query, store, metric) LSH lookup + score + sort; metric is Option<DistanceMetric>
.query(vec) Raw LSH lookup — returns unscored candidate IDs
.len() / .is_empty() Size queries
.clear() Remove all entries

License

MIT