Skip to main content

Crate rig_core

Crate rig_core 

Source
Expand description

Rig is a Rust library for building LLM-powered applications that focuses on ergonomics and modularity.

§Table of contents

§High-level features

  • Full support for LLM completion and embedding workflows
  • Simple but powerful common abstractions over LLM providers (e.g. OpenAI, Cohere) and vector stores (e.g. MongoDB, in-memory)
  • Integrate LLMs in your app with minimal boilerplate

§Simple example

use rig_core::{
    client::{CompletionClient, ProviderClient},
    completion::{AssistantContent, CompletionModel},
    providers::openai,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create an OpenAI client and completion model.
    // This requires the `OPENAI_API_KEY` environment variable to be set.
    let openai_client = openai::Client::from_env()?;
    let model = openai_client.completion_model(openai::GPT_5_2);

    let request = model.completion_request("Who are you?").build();
    let response = model.completion(request).await?;
    for item in response.choice {
        if let AssistantContent::Text(text) = item {
            println!("{}", text.text);
        }
    }

    Ok(())
}

Note: using #[tokio::main] requires you enable tokio’s macros and rt-multi-thread features or just full to enable all features (cargo add tokio --features macros,rt-multi-thread).

§Core concepts

§Completion and embedding models

Rig provides a consistent API for working with LLMs and embeddings. Specifically, each provider (e.g. OpenAI, Cohere) has a Client struct that can be used to initialize completion and embedding models. These models implement the CompletionModel and EmbeddingModel traits respectively, which provide a common, low-level interface for creating completion and embedding requests and executing them.

§Agent runtimes

This crate owns the provider-agnostic model, message, tool, and storage contracts. The sibling rig-agent crate provides the classic builder and run-loop API.

§Vector stores and indexes

Rig provides a common interface for working with vector stores and indexes. Specifically, the library provides the VectorStoreIndex trait, which can be implemented to define vector stores and indices respectively. Indexes can be queried directly by applications or runtimes. For active RAG, expose the index through its blanket PortableTool implementation, or through a custom tool, so the model decides when and how to retrieve. The classic rig-agent runtime can also query indexes from hooks and append the resulting documents to a turn’s extra context.

Indexes can also serve custom architectures that use multiple LLMs or agents.

§Conversation memory

Runtimes can load and persist per-conversation history through the ConversationMemory trait. The classic rig-agent runtime integrates this portable backend contract. The default in-process backend InMemoryConversationMemory is suitable for tests and single-process agents; reusable history-shaping policies (sliding window, token budget) live in the rig-memory companion crate. See examples/agent_with_memory.rs for a runnable end-to-end example.

§Integrations

§Model Providers

Rig natively supports the following completion and embedding model provider integrations:

  • Anthropic
  • Azure OpenAI
  • ChatGPT and GitHub Copilot auth-backed clients
  • Cohere
  • DeepSeek
  • Gemini
  • Groq
  • Hugging Face
  • Hyperbolic
  • Llamafile
  • MiniMax
  • Mira
  • Mistral
  • Moonshot
  • Ollama
  • OpenAI
  • OpenRouter
  • Perplexity
  • Together
  • Voyage AI
  • xAI
  • Xiaomi MiMo
  • Z.ai

You can also implement your own model provider integration by defining types that implement the CompletionModel and EmbeddingModel traits.

Vector stores are available as separate companion-crates:

You can also implement your own vector store integration by defining types that implement the VectorStoreIndex trait.

The following providers are available as separate companion-crates:

Re-exports§

pub use completion::message;
pub use embeddings::Embed;
pub use one_or_many::EmptyListError;
pub use one_or_many::OneOrMany;
pub use schemars;
pub use serde;
pub use serde_json;

Modules§

audio_generationaudio
Everything related to audio generation (ie, Text To Speech). Rig abstracts over a number of different providers using the AudioGenerationModel trait.
client
This module provides traits for defining and creating provider clients. Clients are used to create models for completion, embeddings, etc.
completion
Provider-agnostic completion and chat abstractions.
embeddings
Provider-agnostic embedding abstractions.
http_client
id
Lightweight generation of short, unique, URL-safe identifiers.
image_generationimage
Everything related to core image generation abstractions in Rig. Rig allows calling a number of different providers (that support image generation) using the ImageGenerationModel trait.
loaders
File loading utilities for preparing local documents as model or embedding input.
markers
Common marker traits and structs for type-safe builders.
memory
Conversation memory: Rig-managed persistent conversation history for agents.
model
Model metadata returned by providers with model listing support.
one_or_many
prelude
The rig prelude.
providers
Provider integrations included in rig-core.
rerank
Provider-agnostic reranking abstractions.
streaming
This module provides functionality for working with streaming completion models. It provides traits and types for generating streaming completion requests and handling streaming completion responses.
telemetry
This module primarily concerns being able to orchestrate telemetry across a given pipeline or workflow. This includes tracing, being able to send traces to an OpenTelemetry collector, setting up your agents with the correct tracing style so you can emit the right traces for platforms like Langfuse, and more.
test_utilstest-utils
Test utilities for deterministic completion-model tests.
tool
Portable tool contracts and canonical execution values.
transcription
This module provides functionality for working with audio transcription models. It provides traits, structs, and enums for generating audio transcription requests, handling transcription responses, and defining transcription models.
vector_store
Vector store abstractions for semantic search and retrieval.
wasm_compat

Macros§

completion_parent_span
Declare a completion-parent span conforming to the adoption contract.
if_not_wasm
if_wasm

Structs§

ProviderResponseError
A raw error response preserved from a provider.

Attribute Macros§

rig_toolderive
A procedural macro that transforms a function into a portable rig_core::tool::PortableTool, or into the classic contextual rig::tool::Tool when the function accepts classic runtime context.
tool_macroderive
A procedural macro that transforms a function into a portable rig_core::tool::PortableTool, or into the classic contextual rig::tool::Tool when the function accepts classic runtime context.

Derive Macros§

Embedderive
A macro that allows you to implement the rig::embedding::Embed trait by deriving it. Usage can be found below: