rai-sdk

rai-sdk is a Rust SDK for building backend AI workflows across OpenAI, Anthropic, and OpenRouter. It provides typed model selection, typestate request builders, structured output validation, streaming, retry/backoff, multimodal prompts, and automatic tool execution loops.

• API reference: https://docs.rs/rai-sdk
• Guide: https://rmagatti.github.io/rai-sdk/

Project status: early and pre-1.0. The crate is usable today, but the public API may change in breaking ways before 1.0. Pin an exact version if you need stability.

Features

• Typed providers and models: use Model::gpt4o_mini(), Model::claude_sonnet_46(), Model::openrouter_auto(), or custom provider model IDs.
• Typestate request builders: .generate() is only available after a prompt and model are available at compile time.
• Structured output: derive JsonSchema and call .generate_structured::<T>() or .generate_structured_once::<T>().
• Tool calling: register typed async tools; generate() executes tool calls and feeds results back to the model until a final answer is produced.
• Streaming: consume provider stream events directly, high-level stream events, or use stream_accumulated() to stream internally and return a full response.
• Retry/backoff: transient RateLimit, Timeout, and HTTP errors are retried with configurable exponential backoff and jitter.
• Multimodal prompts: send text, image, audio, video, and file content blocks. Provider support varies.

Installation

cargo add rai-sdk

Or add it to your Cargo.toml directly, along with the crates the examples below use:

[dependencies]
rai-sdk = "0.1"
tokio = { version = "1", features = ["full"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
futures = "0.3"

The minimum supported Rust version is 1.86.

Feature flags

Providers (all enabled by default):

• openai — OpenAI Chat Completions
• anthropic — Anthropic Messages
• openrouter — OpenRouter (aggregates many vendors)

TLS backend (at least one required when a provider is enabled):

• rustls-tls (default) — no system OpenSSL needed, but builds aws-lc-rs, which requires cmake and a C compiler
• native-tls — uses the platform TLS stack and avoids building aws-lc-rs/cmake (Linux needs OpenSSL development files)

Since the TLS backend is part of the default feature set, turning defaults off means naming one explicitly:

[dependencies]
rai-sdk = { version = "0.1", default-features = false, features = ["anthropic", "rustls-tls"] }

Building in a minimal container without cmake? Use native-tls instead:

[dependencies]
rai-sdk = { version = "0.1", default-features = false, features = ["anthropic", "native-tls"] }

Omitting both while enabling a provider fails the build with an explanatory message. A providerless --no-default-features build remains valid. If Cargo feature unification enables both TLS features, rai-sdk uses rustls; use the default-features = false form above to avoid compiling it.

Configuration

Use environment variables:

export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export OPENROUTER_API_KEY="sk-or-..."

Optional provider settings:

export OPENAI_BASE_URL="https://api.openai.com/v1"
export ANTHROPIC_BASE_URL="https://api.anthropic.com"
export OPENROUTER_BASE_URL="https://openrouter.ai/api/v1"
export OPENROUTER_HTTP_REFERER="https://your-app.example"
export OPENROUTER_TITLE="Your App"
export OPENROUTER_CATEGORIES="productivity,agents"
export AI_TIMEOUT_SECONDS="120"

Optional retry settings:

export AI_MAX_RETRIES="3"
export AI_RETRY_INITIAL_DELAY_MS="1000"
export AI_RETRY_MAX_DELAY_MS="60000"
export AI_RETRY_BACKOFF_MULTIPLIER="2.0"
export AI_RETRY_JITTER="true"

You can also configure everything in code with ClientBuilder and RetryConfig.

Basic Chat

use rai_sdk::{ClientBuilder, Model};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = ClientBuilder::new()
        .from_env()
        .model(Model::gpt4o_mini())
        .build()?;

    let response = client
        .request()
        .prompt("Explain Rust ownership in two sentences.")
        .generate()
        .await?;

    println!("{}", response.text());
    Ok(())
}

OpenRouter

use rai_sdk::{ClientBuilder, Model};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = ClientBuilder::new()
        .from_env()
        .openrouter_http_referer("https://your-app.example")
        .openrouter_title("Your App")
        .model(Model::openrouter_auto())
        .build()?;

    let response = client
        .request()
        .prompt("Pick the best model available and summarize OpenRouter in one paragraph.")
        .generate()
        .await?;

    println!("{}", response.text());
    Ok(())
}

Use curated OpenRouter constructors like Model::openrouter_gpt5(), Model::openrouter_deepseek_r1(), and Model::openrouter_qwen3_coder(), or pass any provider model ID with Model::openrouter_custom("vendor/model").

Structured Output

generate_structured() validates the model response against a generated JSON Schema and deserializes it into your Rust type.

use rai_sdk::{ClientBuilder, GenerationConfig, JsonSchema, Model};
use serde::{Deserialize, Serialize};

#[derive(Debug, Deserialize, Serialize, JsonSchema)]
struct Recipe {
    name: String,
    ingredients: Vec<String>,
    steps: Vec<String>,
    prep_time_minutes: u32,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = ClientBuilder::new()
        .from_env()
        .model(Model::gpt4o_mini())
        .build()?;

    let structured = client
        .request()
        .config(GenerationConfig::new().with_temperature(0.2))
        .prompt("Return a simple chocolate cake recipe as JSON.")
        .generate_structured::<Recipe>()
        .await?;

    println!("{:#?}", structured.output);
    Ok(())
}

Use generate_structured_once() when configured tools should be ignored and you want a single provider response.

Tool Calling

Tools are typed handlers. generate() automatically runs tool calls, appends tool results, and asks the model to continue until it returns a final response.

use rai_sdk::{ClientBuilder, JsonSchema, Model, Result, Tool, ToolContext};
use serde::{Deserialize, Serialize};
use serde_json::json;

#[derive(Debug, Deserialize, Serialize, JsonSchema)]
struct WeatherArgs {
    city: String,
    #[serde(default = "default_unit")]
    unit: String,
}

fn default_unit() -> String {
    "celsius".to_string()
}

async fn get_weather(args: WeatherArgs, _ctx: ToolContext) -> Result<serde_json::Value> {
    Ok(json!({
        "city": args.city,
        "temperature": 22,
        "unit": args.unit,
        "condition": "Sunny"
    }))
}

#[tokio::main]
async fn main() -> std::result::Result<(), Box<dyn std::error::Error>> {
    let weather_tool = Tool::new("get_current_weather")
        .description("Get the current weather in a city.")
        .handler(get_weather)?;

    let client = ClientBuilder::new()
        .from_env()
        .model(Model::gpt4o_mini())
        .tool(weather_tool)
        .build()?;

    let response = client
        .request()
        .prompt("What is the weather in Paris right now?")
        .generate()
        .await?;

    println!("{}", response.text());
    Ok(())
}

Use .generate_once() if you want the raw provider response with tool calls but do not want the SDK to execute registered tools.

Streaming

For a complete response assembled from the streaming transport:

use rai_sdk::{ClientBuilder, Model};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = ClientBuilder::new()
        .from_env()
        .model(Model::gpt4o_mini())
        .build()?;

    let response = client
        .request()
        .prompt("Write a short launch announcement.")
        .stream_accumulated()
        .await?;

    println!("{}", response.text());
    Ok(())
}

For raw stream events:

use futures::StreamExt;
use rai_sdk::{provider::ProviderStreamEvent, ClientBuilder, Model};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = ClientBuilder::new()
        .from_env()
        .model(Model::gpt4o_mini())
        .build()?;

    let mut stream = client
        .request()
        .prompt("Count from one to five.")
        .stream()
        .await?;

    while let Some(event) = stream.next().await {
        match event? {
            ProviderStreamEvent::Text(text) => print!("{text}"),
            ProviderStreamEvent::Done { .. } => println!(),
            _ => {}
        }
    }

    Ok(())
}

Multimodal Prompt

use rai_sdk::{ClientBuilder, ContentBlock, Message, Model, Prompt};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = ClientBuilder::new()
        .from_env()
        .model(Model::gpt4o_mini())
        .build()?;

    let prompt = Prompt::single(Message::user_multimodal(vec![
        ContentBlock::text("Describe this image in one sentence."),
        ContentBlock::image_url("https://example.com/image.png"),
    ]));

    let response = client.request().prompt(prompt).generate().await?;
    println!("{}", response.text());
    Ok(())
}

OpenAI and OpenRouter currently serialize image content. Other block types are represented in the common prompt model, but provider-specific support may be incomplete.

Retry Configuration

use std::time::Duration;

use rai_sdk::{ClientBuilder, Model, RetryConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let retry = RetryConfig::new()
        .with_max_retries(5)
        .with_initial_delay(Duration::from_millis(500))
        .with_max_delay(Duration::from_secs(30))
        .with_jitter(true);

    let client = ClientBuilder::new()
        .from_env()
        .model(Model::claude_sonnet_46())
        .retry_config(retry)
        .build()?;

    let response = client
        .request()
        .prompt("Give me three practical Rust error-handling tips.")
        .generate()
        .await?;

    println!("{}", response.text());
    Ok(())
}

Disable retries globally with ClientBuilder::new().no_retry() or per request with .request().no_retry().

Examples

Run bundled examples from this repository:

cargo run --example basic_chat
cargo run --example structured_output
cargo run --example tool_calling

Notes

• generate() auto-executes registered tools. generate_once() does not.
• generate_structured() may use tools before producing typed output. generate_structured_once() ignores configured tools.
• Streaming with registered tools is intentionally rejected by the raw streaming API.
• Provider availability is based on enabled Cargo features and configured credentials.

Documentation

• API reference on docs.rs — every public type and method.
• Guide — task-oriented chapters on configuration, providers, structured output, tool calling, streaming, and retries.

Contributing

Contributions are welcome. See CONTRIBUTING.md for local setup, the commands CI runs, and the testing policy — the test suite is fully offline and must never require API credentials.

Please also read our Code of Conduct. To report a security issue, follow SECURITY.md rather than opening a public issue.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.