genai 0.6.0-beta.21

Multi-AI Providers Library for Rust. (OpenAI, Gemini, Anthropic, xAI, Ollama, Groq, DeepSeek, Grok, GitHub Copilot)
Documentation

genai, Native-Protocol Multi-AI Provider Library for Rust

Provides a single, ergonomic Rust API for native-protocol multi-AI provider access, including Anthropic, OpenAI, Gemini, xAI, Ollama, Groq, and more.

Currently natively supports over 25 providers: openai, openai_resp, anthropic, gemini, xai, ollama, ollama_cloud, opencode_go, groq, deepseek, cohere, together, fireworks, nebius, mimo, zai (Zhipu AI), bigmodel, aliyun, baidu, moonshot, vertex, github_copilot (GitHub Models API), aihubmix, bedrock_api, bedrock_sigv4, open_router.

Also supports a custom Endpoint and Auth with ServiceTargetResolver (see examples/c06-target-resolver.rs).

NOTE: Use genai = "0.6.0-beta.21" or later for improved robustness, even compared to 0.5.x, along with many more providers, fixes, performance improvements, and API enhancements. v0.6.0 is coming soon.

Docs for LLMs | CHANGELOG | BIG THANKS

Model to Adapter Resolution

By default, the library resolves the AdapterKind (AI Provider) based on the model name prefix:

  • OpenAI: gpt-* (most), o1-*, o3-*, o4-*, chatgpt-*, codex-*
  • OpenAI Responses: gpt-5-*, gpt-* (containing codex or pro)
  • Anthropic: claude-*
  • Gemini: gemini-*
  • xAI: grok-*
  • DeepSeek: deepseek-*
  • Moonshot: moonshot-*
  • Zai: glm-*
  • Cohere: command-*, embed-*
  • Mimo: mimo-*
  • OpenCode Go: Namespace opencode_go:: only
  • Fireworks: Models containing fireworks
  • Ollama: Fallback for any other names, defaulting to local Ollama.

Namespacing (Forcing an Adapter)

You can force a specific adapter by using the adapter_kind::model_name syntax. This is the recommended way for many providers and for disambiguating OpenAI-compatible services.

  • groq::llama-3.1-8b-instant (Forces Groq adapter)
  • together::meta-llama/Llama-3-8b-chat-hf (Forces Together adapter)
  • ollama_cloud::gemma3:4b (Forces OllamaCloud adapter)
  • github_copilot::openai/gpt-4.1-mini (Forces GithubCopilot adapter)
  • nebius::Qwen/Qwen3-235B-A22B (Forces Nebius adapter)
  • aliyun::qwen-plus (Forces Aliyun adapter)
  • vertex::gemini-2.5-flash (Forces Google Vertex adapter)
  • moonshot::moonshot-v1-8k (Forces Moonshot adapter)
  • baidu::ernie-4.0 (Forces Baidu adapter)
  • coding::glm-4.6 (Special namespace for Zai coding subscription)
  • opencode_go::minimax-m2.5 (Forces OpenCodeGo adapter)
  • bedrock_api::anthropic.claude-v2 (Forces AWS Bedrock adapter)
  • open_router::google/gemini-2.0-flash-001 (Forces OpenRouter adapter)

For a complete list of AdapterKind, see the AdapterKind enum.

v0.6.x - (coming soon, available as v0.6.0-beta.21)

  • What's new:
    • New Adapters:
      • AWS Bedrock (bedrock_api and bedrock_sigv4 adapters)
      • open_router
      • vertex (with Gemini and Anthropic support)
      • github_copilot (GitHub Models API)
      • opencode_go
      • baidu
      • aliyun
      • moonshot
      • aihubmix
      • ollama_cloud (Ollama Cloud)
    • Expanded Provider Support: Comprehensive coverage of major AI ecosystems.
    • Updated API: Refined ReasoningContent and StopReason handling (v0.6.0-beta.20), including ContentPart::ReasoningContent and provider stop reasons.
    • ProviderConfig for model listing: Client::all_model_names(adapter_kind, provider_config) now accepts endpoint and auth overrides, including remote Ollama hosts and custom OpenAI-compatible model listing.
    • Bound adapter clients: ClientBuilder::with_adapter_kind(...) and ClientConfig::with_adapter_kind(...) bind a client to a single provider adapter, useful for proxies, gateways, Azure-style deployment names, and OpenAI-compatible providers with non-standard model names.
    • ModelSpec and ServiceTarget: Model arguments can be represented as a model name, explicit ModelIden, or complete ServiceTarget, enabling custom endpoint, auth, and model identity without relying on model-name inference.
    • OpenAI Responses stateful sessions: OpenAI Responses supports session continuity with previous_response_id, request store, and returned response_id.
    • Chat extra body: ChatOptions::with_extra_body(...) provides a low-level request body extension point for provider-specific fields in OpenAI-compatible chat payloads.
    • Tool choice: ChatOptions::with_tool_choice(...) adds provider-neutral tool selection hints for automatic, disabled, required, or specific tool calls.
    • Built-in tools and WebSearch: Added typed built-in tool support, including ToolName, ToolConfig, WebSearch, and provider mappings for Anthropic, OpenAI, and Gemini.
    • Prompt cache controls: Chat-level CacheControl support adds provider-specific prompt caching options, including OpenAI prompt_cache_key and cache retention.
    • Reasoning effort additions: Added ReasoningEffort::Max for Anthropic and ReasoningEffort::XHigh for OpenAI.
    • Gemini schema compatibility: Gemini and Vertex Gemini structured output and tool schemas now normalize common JSON Schema shapes, including const, nullable schema patterns, additionalProperties, and JSON Schema-only keywords rejected by Vertex.
    • Ollama and Ollama Cloud: Now Ollama native API protocol
    • Perf Improvements: HTTP requests use performance optimizations such as gzip, TCP_NODELAY, and HTTP/2 tuning.
    • Numerous fixes, optimizations, and API enhancements.

v0.5.x - (2026-01-09 onwards)

  • What's new:
    • New Adapters: BigModel.cn and the MIMO model adapter (thanks to Akagi201).
    • zai: updated namespace strategy, using zai:: for default and zai-coding:: for subscriptions (same adapter).
    • Gemini Thinking & Thought: Full support for Gemini Thought signatures (thanks to Himmelschmidt) and thinking levels.
    • Reasoning Effort Control: Support for ReasoningEffort for Anthropic (Claude 3.7/4.5) and Gemini (Thinking levels), including ReasoningEffort::None.
    • Content & Binary Improvements: Enhanced binary/PDF API and size tracking.
    • Internal Stream Refactor: Switched to a unified EventSourceStream and WebStream for better reliability and performance across all providers.
    • Dependency Upgrade: Now using reqwest 0.13.
  • Core Features:
    • Normalized and ergonomic Chat API across all major providers.
    • Native protocol support for Gemini and Anthropic protocols (Reasoning/Thinking controls).
    • PDF, image, and embedding support.
    • Custom authentication, endpoint, and header overrides.

See CHANGELOG

Usage examples

  • Check out AIPACK, which wraps this genai library into an agentic runtime to run, build, and share AI Agent Packs. See pro@coder for a simple example of how I use AI PACK/genai for production coding.

Note: Feel free to send me a short description and a link to your application or library that uses genai.

Key Features

  • Multi-AI Provider/Model access optimized per provider: native protocols when available, OpenAI-compatible APIs when appropriate or required, and one common Rust API for OpenAI, OpenAI Responses, Anthropic, Gemini, Ollama, Ollama Cloud, OpenCode Go, Groq, xAI, DeepSeek, Cohere, Together, Fireworks, Nebius, Mimo, Zai, BigModel, Aliyun, Google Vertex, and GitHub Copilot (direct chat and streaming) (see examples/c00-readme.rs)
  • DeepSeekR1 support, with reasoning_content (and stream support), plus DeepSeek Groq and Ollama support (and reasoning_content normalization)
  • Image Analysis (for OpenAI, Gemini flash-2, Anthropic) (see examples/c07-image.rs)
  • Custom Auth/API Key (see examples/c02-auth.rs)
  • Model aliases (see examples/c05-model-names.rs)
  • Custom endpoint, auth, and model identifier (see examples/c06-target-resolver.rs)

Examples | Thanks | Library Focus | Changelog | Provider Mapping: ChatOptions | Usage

Examples

examples/c00-readme.rs

//! Base examples demonstrating the core capabilities of genai

use genai::chat::printer::{print_chat_stream, PrintChatStreamOptions};
use genai::chat::{ChatMessage, ChatRequest};
use genai::Client;

const MODEL_OPENAI: &str = "gpt-4o-mini"; // o1-mini, gpt-4o-mini
const MODEL_ANTHROPIC: &str = "claude-3-haiku-20240307";
// or namespaced with simple name "fireworks::qwen3-30b-a3b", or "fireworks::accounts/fireworks/models/qwen3-30b-a3b"
const MODEL_FIREWORKS: &str = "accounts/fireworks/models/qwen3-30b-a3b";
const MODEL_TOGETHER: &str = "together::openai/gpt-oss-20b";
const MODEL_GEMINI: &str = "gemini-2.0-flash";
const MODEL_GROQ: &str = "groq::llama-3.1-8b-instant";
const MODEL_OLLAMA: &str = "gemma:2b"; // sh: `ollama pull gemma:2b`
const MODEL_OLLAMA_CLOUD: &str = "ollama_cloud::gemma3:4b";
const MODEL_XAI: &str = "grok-3-mini";
const MODEL_DEEPSEEK: &str = "deepseek-chat";
const MODEL_ZAI: &str = "glm-4-plus";
const MODEL_COHERE: &str = "command-r7b-12-2024";
const MODEL_MOONSHOT: &str = "moonshot::moonshot-v1-8k";
const MODEL_BAIDU: &str = "baidu::ernie-4.0";
const MODEL_BIGMODEL: &str = "bigmodel::glm-4-plus";
const MODEL_ALIYUN: &str = "aliyun::qwen-plus";
// or any publisher: "github_copilot::anthropic/claude-sonnet-4-6", "github_copilot::google/gemini-2.5-pro", "github_copilot::xai/grok-3-mini"
const MODEL_GITHUB_COPILOT: &str = "github_copilot::openai/gpt-4.1-mini";
const MODEL_OPEN_ROUTER: &str = "open_router::google/gemini-2.0-flash-001";

// NOTE: These are the default environment keys for each AI Adapter Type.
//       They can be customized; see `examples/c02-auth.rs`.
const MODEL_AND_KEY_ENV_NAME_LIST: &[(&str, &str)] = &[
	// -- De/activate models/providers
	(MODEL_OPENAI, "OPENAI_API_KEY"),
	(MODEL_ANTHROPIC, "ANTHROPIC_API_KEY"),
	(MODEL_GEMINI, "GEMINI_API_KEY"),
	(MODEL_FIREWORKS, "FIREWORKS_API_KEY"),
	(MODEL_TOGETHER, "TOGETHER_API_KEY"),
	(MODEL_GROQ, "GROQ_API_KEY"),
	(MODEL_XAI, "XAI_API_KEY"),
	(MODEL_DEEPSEEK, "DEEPSEEK_API_KEY"),
	(MODEL_OLLAMA, ""),
	(MODEL_OLLAMA_CLOUD, "OLLAMA_API_KEY"),
	(MODEL_ZAI, "ZAI_API_KEY"),
	(MODEL_COHERE, "COHERE_API_KEY"),
	(MODEL_MOONSHOT, "MOONSHOT_API_KEY"),
	(MODEL_BAIDU, "BAIDU_API_KEY"),
	(MODEL_BIGMODEL, "BIGMODEL_API_KEY"),
	(MODEL_ALIYUN, "ALIYUN_API_KEY"),
	(MODEL_GITHUB_COPILOT, "GITHUB_TOKEN"),
	(MODEL_OPEN_ROUTER, "OPEN_ROUTER_API_KEY"),
];

// NOTE: Model to AdapterKind (AI Provider) type mapping rule
//  - starts_with "gpt"      -> OpenAI (or OpenAI Responses for gpt-5/codex/pro)
//  - starts_with "claude"   -> Anthropic
//  - starts_with "command"  -> Cohere
//  - starts_with "gemini"   -> Gemini
//  - model in Groq models   -> Groq
//  - starts_with "glm"      -> ZAI
//  - starts_with "ollama_cloud::" -> OllamaCloud
//  - For anything else      -> Ollama
//
// This can be customized; see `examples/c03-mapper.rs`

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
	let question = "Why is the sky red?";

	let chat_req = ChatRequest::new(vec![
		// -- Messages (de/activate to see the differences)
		ChatMessage::system("Answer in one sentence"),
		ChatMessage::user(question),
	]);

	let client = Client::default();

	let print_options = PrintChatStreamOptions::from_print_events(false);

	for (model, env_name) in MODEL_AND_KEY_ENV_NAME_LIST {
		// Skip if the environment name is not set
		if !env_name.is_empty() && std::env::var(env_name).is_err() {
			println!("===== Skipping model: {model} (env var not set: {env_name})");
			continue;
		}

		let adapter_kind = client.resolve_service_target(model).await?.model.adapter_kind;

		println!("\n===== MODEL: {model} ({adapter_kind}) =====");

		println!("\n--- Question:\n{question}");

		println!("\n--- Answer:");
		let chat_res = client.exec_chat(model, chat_req.clone(), None).await?;
		println!("{}", chat_res.first_text().unwrap_or("NO ANSWER"));

		println!("\n--- Answer: (streaming)");
		let chat_res = client.exec_chat_stream(model, chat_req.clone(), None).await?;
		print_chat_stream(chat_res, Some(&print_options)).await?;

		println!();
	}

	Ok(())
}

More Examples

Library Focus:

  • Focuses on standardizing chat completion APIs across major AI providers while preserving provider-specific strengths.

  • Native implementation without per-service SDK dependencies.

    • Reason: genai uses each provider's native protocol when available, so features such as reasoning controls, thinking budgets, streaming metadata, and multimodal inputs can be represented more completely. When a provider primarily exposes an OpenAI-compatible API, genai uses that compatibility layer instead. Managing these protocol differences at the adapter layer is simpler and more cumulative than dealing with multiple SDKs.

  • Prioritizes ergonomics and commonality, while depth is secondary. (If you require a complete client API, consider using async-openai and ollama-rs; both are excellent and easy to use.)

  • This library focuses on text chat APIs, with vision and function calling support being expanded.

ChatOptions

  • (1) - OpenAI-compatible notes
    • Models: OpenAI, DeepSeek, Groq, Ollama, xAI, Mimo, Together, Fireworks, Nebius, Zai, AIHubMix
Property OpenAI Compatibles (*1) Anthropic Gemini generationConfig. Cohere
temperature temperature temperature temperature temperature
max_tokens max_tokens max_tokens (default 1024) maxOutputTokens max_tokens
top_p top_p top_p topP p

Usage

Property OpenAI Compatibles (1) Anthropic usage. Gemini usageMetadata. Cohere meta.tokens.
prompt_tokens prompt_tokens input_tokens (added) promptTokenCount (2) input_tokens
completion_tokens completion_tokens output_tokens (added) candidatesTokenCount (2) output_tokens
total_tokens total_tokens (computed) totalTokenCount (2) (computed)
prompt_tokens_details prompt_tokens_details cached/cache_creation N/A for now N/A for now
completion_tokens_details completion_tokens_details N/A for now N/A for now N/A for now

  • (1) - OpenAI-compatible notes

  • (2): Gemini tokens

    • Right now, with the Gemini Stream API, it's not clear whether usage for each event is cumulative or must be summed. It appears to be cumulative, meaning the last message shows the total number of input, output, and total tokens, so that is the current assumption. See possible tweet answer for more info.

Links