adk-model

LLM model integrations for Rust Agent Development Kit (ADK-Rust) with Gemini, OpenAI, OpenRouter, xAI, Anthropic, DeepSeek, Groq, Ollama, Fireworks AI, Together AI, Mistral AI, Perplexity, Cerebras, SambaNova, Amazon Bedrock, and Azure AI Inference.

Overview

adk-model provides LLM integrations for the Rust Agent Development Kit (ADK-Rust). Supports all major providers:

Gemini - Google's Gemini models (3 Pro, 3 Flash, 2.5 Pro, 2.5 Flash, etc.)
OpenAI - GPT-5.1, GPT-5, GPT-5 Mini, GPT-4o (legacy)
OpenRouter - Native chat, responses, routing, discovery, and credits APIs
xAI - Grok models through the OpenAI-compatible API
Anthropic - Claude Opus 4.8, Claude Opus 4.7, Claude Sonnet 4.6, Claude Haiku 4.5
DeepSeek - DeepSeek R1, DeepSeek V3.1, DeepSeek-Chat with thinking mode
Groq - Ultra-fast inference (LLaMA 3.3, Mixtral, Gemma)
Ollama - Local LLMs (LLaMA, Mistral, Qwen, Gemma, etc.)
Fireworks AI - Fast open-model inference (Llama, Mixtral, etc.)
Together AI - Hosted open models (Llama, CodeLlama, etc.)
Mistral AI - Mistral cloud models (Mistral Small, Large, etc.)
Perplexity - Search-augmented LLM (Sonar, etc.)
Cerebras - Ultra-fast inference (Llama 3.3, etc.)
SambaNova - Fast inference (Llama 3.3, etc.)
Amazon Bedrock - AWS-hosted models via IAM auth (Claude, Llama, Mistral, etc.)
Azure AI Inference - Azure-hosted models (Cohere, Llama, Mistral, etc.)
Streaming - Real-time response streaming for all providers
Multimodal - Text, images, audio, video, and PDF input

The crate implements the Llm trait from adk-core, allowing models to be used interchangeably.

Installation

[dependencies]
adk-model = "2.1.0"

Enable provider-specific features as needed:

[dependencies]
adk-model = { version = "2.1.0", features = ["openrouter"] }

Or use the meta-crate:

[dependencies]
adk-rust = { version = "2.1.0", features = ["models"] }

Quick Start

Gemini (Google)

use adk_model::GeminiModel;
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("GOOGLE_API_KEY")?;
    let model = GeminiModel::new(&api_key, "gemini-3.1-pro-preview")?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

GeminiModel::from_env(model) builds the client from the environment: a truthy GOOGLE_GENAI_USE_ENTERPRISE or GOOGLE_GENAI_USE_VERTEXAI (1 or a case-insensitive true) selects Vertex AI via Application Default Credentials using GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION (requires the gemini-vertex feature); otherwise the Gemini API via GOOGLE_API_KEY or GEMINI_API_KEY.

OpenAI

use adk_model::openai::{OpenAIClient, OpenAIConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("OPENAI_API_KEY")?;
    let model = OpenAIClient::new(OpenAIConfig::new(api_key, "gpt-5.6-terra"))?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

OpenRouter

use adk_core::{Content, Llm, LlmRequest};
use adk_model::openrouter::{OpenRouterClient, OpenRouterConfig};
use futures::StreamExt;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("OPENROUTER_API_KEY")?;
    let client = OpenRouterClient::new(
        OpenRouterConfig::new(api_key, "qwen/qwen3.7-max")
            .with_http_referer("https://github.com/zavora-ai/adk-rust")
            .with_title("ADK-Rust"),
    )?;

    let request = LlmRequest::new(
        "qwen/qwen3.7-max",
        vec![Content::new("user").with_text("Reply in one short sentence.")],
    );
    let mut stream = client.generate_content(request, true).await?;

    while let Some(item) = stream.next().await {
        let _response = item?;
    }

    Ok(())
}

For native OpenRouter discovery and provider features, call OpenRouterClient directly:

use adk_model::openrouter::{OpenRouterClient, OpenRouterConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("OPENROUTER_API_KEY")?;
    let client = OpenRouterClient::new(
        OpenRouterConfig::new(api_key, "qwen/qwen3.7-max")
            .with_http_referer("https://github.com/zavora-ai/adk-rust")
            .with_title("ADK-Rust"),
    )?;

    let models = client.list_models().await?;
    let credits = client.get_credits().await?;

    println!("models: {}", models.data.len());
    println!("credits remaining: {}", credits.data.total_credits - credits.data.total_usage);
    Ok(())
}

OpenRouter Scope Boundary

Use the native OpenRouterClient APIs when you need:

model discovery or credits
exact chat or responses payloads
OpenRouter routing, fallback, provider preferences, or plugins
built-in tools such as web search
raw provider metadata and annotations

Use the generic Llm adapter when you need OpenRouter inside ADK agents and runners. The adapter supports streaming, tool calling, reasoning parts, multimodal requests, and OpenRouter-specific request tuning through OpenRouterRequestOptions, but it intentionally maps provider-native responses into the generic LlmRequest and LlmResponse shape.

For live end-to-end validation and agentic examples, see:

examples/openrouter
adk-model/examples/openrouter_chat.rs
adk-model/examples/openrouter_responses.rs
adk-model/examples/openrouter_adapter.rs
adk-model/examples/openrouter_discovery.rs

OpenAI Responses API

The Responses API (/v1/responses) is OpenAI's recommended endpoint for their latest models, including reasoning models with summaries and built-in tools.

use adk_model::openai::{OpenAIResponsesClient, OpenAIResponsesConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("OPENAI_API_KEY")?;
    let config = OpenAIResponsesConfig::new(api_key, "gpt-5.6-terra");
    let model = OpenAIResponsesClient::new(config)?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

For reasoning models with summaries:

use adk_model::openai::{
    OpenAIResponsesClient, OpenAIResponsesConfig,
    ReasoningEffort, ReasoningSummary,
};

let config = OpenAIResponsesConfig::new(api_key, "o4-mini")
    .with_reasoning_effort(ReasoningEffort::Medium)
    .with_reasoning_summary(ReasoningSummary::Detailed);
let model = OpenAIResponsesClient::new(config)?;

See the full Responses API documentation and the examples/openai_responses/ example for a complete 7-scenario demo.

Background Mode & Cancellation

For long-running requests (deep research, complex reasoning), use background mode and polling:

// Submit with background: true
let mut gen_config = GenerateContentConfig::default();
gen_config.extensions.insert("openai".into(), serde_json::json!({ "background": true }));

// Poll for completion
let response = client.poll_response("resp_abc123").await?;

// Cancel a running background response
let cancelled = client.cancel_response("resp_abc123").await?;

Additional Responses API Examples

Six standalone example crates demonstrate specific features:

Example	Feature
`examples/openai_ws_minimal/`	WebSocket transport (low-latency)
`examples/openai_background/`	Background mode (submit & poll)
`examples/openai_conversations/`	Conversations API lifecycle
`examples/openai_builtin_tools/`	Built-in tools (image gen, web search)
`examples/openai_deep_research/`	Deep research models (auto-background)
`examples/openai_open_responses/`	Open Responses (provider-agnostic)

OpenAI Reasoning Effort

For current reasoning models, use the complete OpenAI reasoning vocabulary:

use adk_model::openai::{
    OpenAIReasoningEffort, OpenAIResponsesClient, OpenAIResponsesConfig,
};

let config = OpenAIResponsesConfig::new(api_key, "gpt-5.6-terra");
let model = OpenAIResponsesClient::new_with_reasoning_effort(
    config,
    OpenAIReasoningEffort::Max,
)?;

Available values are None, Minimal, Low, Medium, High, XHigh, and Max. Model and API support varies: GPT-5.6 supports every value except Minimal through the Responses API, while Chat Completions supports up to XHigh. The original ReasoningEffort::{Low, Medium, High} configuration remains available for backward compatibility.

Anthropic (Claude)

use adk_model::anthropic::{AnthropicClient, AnthropicConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("ANTHROPIC_API_KEY")?;
    let model = AnthropicClient::new(AnthropicConfig::new(api_key, "claude-sonnet-5"))?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Anthropic Advanced Features

use adk_model::anthropic::{AnthropicClient, AnthropicConfig, ThinkingMode};

// Current Claude models use adaptive thinking rather than token budgets.
let config = AnthropicConfig::new(api_key, "claude-sonnet-5")
    .with_thinking_mode(ThinkingMode::Adaptive)
    .with_prompt_caching(true)
    .with_beta_feature("prompt-caching-2024-07-31");
let client = AnthropicClient::new(config)?;

// Token counting
let count = client.count_tokens(&request).await?;

// Model discovery
let models = client.list_models().await?;
let info = client.get_model("claude-sonnet-5").await?;

// Rate limit inspection
let rate_info = client.latest_rate_limit_info().await;

DeepSeek

use adk_model::deepseek::{DeepSeekClient, DeepSeekConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("DEEPSEEK_API_KEY")?;

    // Standard chat model
    let model = DeepSeekClient::chat(api_key)?;

    // Or use the reasoner model with chain-of-thought
    // let model = DeepSeekClient::reasoner(api_key)?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Groq (Ultra-Fast)

use adk_model::groq::{GroqClient, GroqConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("GROQ_API_KEY")?;
    let model = GroqClient::new(GroqConfig::gpt_oss_120b(api_key))?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Ollama (Local)

use adk_model::ollama::{OllamaModel, OllamaConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Requires: ollama serve && ollama pull llama3.2
    let model = OllamaModel::new(OllamaConfig::new("llama3.2"))?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Fireworks AI

use adk_model::fireworks::{FireworksClient, FireworksConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("FIREWORKS_API_KEY")?;
    let model = FireworksClient::new(FireworksConfig::new(
        api_key, "accounts/fireworks/models/llama-v3p1-8b-instruct",
    ))?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Together AI

use adk_model::together::{TogetherClient, TogetherConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("TOGETHER_API_KEY")?;
    let model = TogetherClient::new(TogetherConfig::new(
        api_key, "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    ))?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Mistral AI

use adk_model::mistral::{MistralClient, MistralConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("MISTRAL_API_KEY")?;
    let model = MistralClient::new(MistralConfig::new(api_key, "mistral-small-latest"))?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Perplexity

use adk_model::perplexity::{PerplexityClient, PerplexityConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("PERPLEXITY_API_KEY")?;
    let model = PerplexityClient::new(PerplexityConfig::new(api_key, "sonar-pro"))?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Cerebras

use adk_model::cerebras::{CerebrasClient, CerebrasConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("CEREBRAS_API_KEY")?;
    let model = CerebrasClient::new(CerebrasConfig::new(api_key, "gpt-oss-120b"))?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

SambaNova

use adk_model::sambanova::{SambaNovaClient, SambaNovaConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("SAMBANOVA_API_KEY")?;
    let model = SambaNovaClient::new(SambaNovaConfig::new(api_key, "gpt-oss-120b"))?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Amazon Bedrock

use adk_model::bedrock::{BedrockClient, BedrockConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Uses AWS IAM credentials. Model IDs are account- and region-specific.
    let model_id = std::env::var("BEDROCK_MODEL")?;
    let config = BedrockConfig::new("us-east-1", model_id);
    let model = BedrockClient::new(config).await?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Azure AI Inference

use adk_model::azure_ai::{AzureAIClient, AzureAIConfig};
use adk_agent::LlmAgentBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("AZURE_AI_API_KEY")?;
    let config = AzureAIConfig::new(
        "https://my-endpoint.eastus.inference.ai.azure.com",
        api_key,
        "meta-llama-3.1-8b-instruct",
    );
    let model = AzureAIClient::new(config)?;

    let agent = LlmAgentBuilder::new("assistant")
        .model(Arc::new(model))
        .build()?;

    Ok(())
}

Supported Models

Google Gemini

Model	Description
`gemini-3.7-flash`	Current balanced default (1M context)
`gemini-3.6-flash`	Previous balanced generation (1M context)
`gemini-3.5-flash-lite`	Cost-efficient high-volume model (1M context)
`gemini-3.1-pro-preview`	Preview reasoning model (2M context)

See Gemini models documentation for the full list.

OpenAI

Model	Description
`gpt-5.6-terra`	Balanced default for agents
`gpt-5.6-sol`	Flagship reasoning and coding
`gpt-5.6-luna`	Cost-efficient high-volume model

See OpenAI models documentation for the full list.

Anthropic Claude

Model	Description
`claude-sonnet-5`	Balanced production default (1M context)
`claude-opus-5`	Flagship autonomous-agent capability (1M context)
`claude-fable-5`	Premium creative and long-form work (1M context)
`claude-haiku-4-5`	Cost-efficient previous generation

See Anthropic models documentation for the full list.

DeepSeek

Model	Description
`deepseek-v4-flash`	Fast balanced default
`deepseek-v4-pro`	Advanced reasoning

Features:

Thinking Mode - Chain-of-thought reasoning with <thinking> tags
Context Caching - Automatic KV cache for repeated prefixes (10x cost reduction)
Tool Calling - Full function calling support

See DeepSeek API documentation for the full list.

Groq

Model	Description
`openai/gpt-oss-120b`	Production default via Groq LPU
`openai/gpt-oss-20b`	Lower-latency economy model

Features:

Ultra-Fast - LPU-based inference (fastest in the industry)
Tool Calling - Full function calling support
Large Context - Up to 128K tokens

See Groq documentation for the full list.

Ollama (Local)

Model	Description
`llama3.3:70b`	Llama 3.3 70B - Latest for local deployment (128K context)
`llama3.2:3b`	Efficient small model
`llama3.1:8b`	Popular balanced model
`deepseek-r1:14b`	Distilled reasoning model
`deepseek-r1:32b`	Larger distilled reasoning model
`qwen3:14b`	Strong multilingual and coding
`qwen2.5:7b`	Efficient multilingual model (recommended for tool calling)
`mistral:7b`	Fast and capable
`mistral-nemo:12b`	Enhanced Mistral variant (128K context)
`gemma3:9b`	Google's efficient open model
`devstral:24b`	Optimized for coding tasks
`codellama:13b`	Code-focused Llama variant

Features:

Local Inference - No API key required
Privacy - Data stays on your machine
Tool Calling - Full function calling support (uses non-streaming for reliability)
MCP Integration - Connect to MCP servers for external tools

See Ollama library for all available models.

New Providers

Provider	Feature Flag	Default Model	API Key Env Var
Fireworks AI	`openai` preset	`accounts/fireworks/models/kimi-k2p6`	`FIREWORKS_API_KEY`
Together AI	`openai` preset	`MiniMaxAI/MiniMax-M2.7`	`TOGETHER_API_KEY`
Mistral AI	`openai` preset	`mistral-medium-latest`	`MISTRAL_API_KEY`
Perplexity	`openai` preset	`sonar-pro`	`PERPLEXITY_API_KEY`
Cerebras	`openai` preset	`gpt-oss-120b`	`CEREBRAS_API_KEY`
SambaNova	`openai` preset	`gpt-oss-120b`	`SAMBANOVA_API_KEY`
Amazon Bedrock	`bedrock`	account/region-specific	AWS IAM credentials
Azure AI Inference	`azure-ai`	(endpoint-specific)	`AZURE_AI_API_KEY`

Features

Streaming - Real-time response streaming for all providers
Tool Calling - Function calling support across all providers
Async - Full async/await support with backpressure
Retry - Automatic retry with exponential backoff
Generation Config - Temperature, top_p, top_k, max_tokens
Token Usage Telemetry - Automatic gen_ai.usage.* span recording for all providers via adk-telemetry

Environment Variables

# Google Gemini
GOOGLE_API_KEY=your-google-api-key

# OpenAI
OPENAI_API_KEY=your-openai-api-key

# xAI
XAI_API_KEY=your-xai-api-key

# Anthropic
ANTHROPIC_API_KEY=your-anthropic-api-key

# DeepSeek
DEEPSEEK_API_KEY=your-deepseek-api-key

# Groq
GROQ_API_KEY=your-groq-api-key

# Fireworks AI
FIREWORKS_API_KEY=your-fireworks-api-key

# Together AI
TOGETHER_API_KEY=your-together-api-key

# Mistral AI
MISTRAL_API_KEY=your-mistral-api-key

# Perplexity
PERPLEXITY_API_KEY=your-perplexity-api-key

# Cerebras
CEREBRAS_API_KEY=your-cerebras-api-key

# SambaNova
SAMBANOVA_API_KEY=your-sambanova-api-key

# Azure AI Inference
AZURE_AI_API_KEY=your-azure-ai-api-key

# Amazon Bedrock (uses AWS IAM credentials)
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_REGION=us-east-1

# Ollama (no key needed, just start the server)
# ollama serve

Feature Flags

Enable specific providers with feature flags:

[dependencies]
# All providers (default)
adk-model = { version = "2.1.0", features = ["all-providers"] }

# Individual providers
adk-model = { version = "2.1.0", features = ["gemini"] }
adk-model = { version = "2.1.0", features = ["openai"] }
adk-model = { version = "2.1.0", features = ["xai"] }
adk-model = { version = "2.1.0", features = ["anthropic"] }
adk-model = { version = "2.1.0", features = ["deepseek"] }
adk-model = { version = "2.1.0", features = ["groq"] }
adk-model = { version = "2.1.0", features = ["ollama"] }
adk-model = { version = "2.1.0", features = ["fireworks"] }
adk-model = { version = "2.1.0", features = ["together"] }
adk-model = { version = "2.1.0", features = ["mistral"] }
adk-model = { version = "2.1.0", features = ["perplexity"] }
adk-model = { version = "2.1.0", features = ["cerebras"] }
adk-model = { version = "2.1.0", features = ["sambanova"] }
adk-model = { version = "2.1.0", features = ["bedrock"] }
adk-model = { version = "2.1.0", features = ["azure-ai"] }

Related Crates

adk-rust - Meta-crate with all components
adk-core - Core Llm trait
adk-agent - Agent implementations

License

Apache-2.0

Part of ADK-Rust

This crate is part of the ADK-Rust framework for building AI agents in Rust.

adk-model 2.1.0

adk-model

Overview

Installation

Quick Start

Gemini (Google)

OpenAI

OpenRouter

OpenRouter Scope Boundary

OpenAI Responses API

Background Mode & Cancellation

Additional Responses API Examples

OpenAI Reasoning Effort

Anthropic (Claude)

Anthropic Advanced Features

DeepSeek

Groq (Ultra-Fast)

Ollama (Local)

Fireworks AI

Together AI

Mistral AI

Perplexity

Cerebras

SambaNova

Amazon Bedrock

Azure AI Inference

Supported Models

Google Gemini

OpenAI

Anthropic Claude

DeepSeek

Groq

Ollama (Local)

New Providers

Features

Environment Variables

Feature Flags

Related Crates

License

Part of ADK-Rust