Crate rig

Source
Expand description

Rig is a Rust library for building LLM-powered applications that focuses on ergonomics and modularity.

§Table of contents

§High-level features

  • Full support for LLM completion and embedding workflows
  • Simple but powerful common abstractions over LLM providers (e.g. OpenAI, Cohere) and vector stores (e.g. MongoDB, in-memory)
  • Integrate LLMs in your app with minimal boilerplate

§Simple example:

use rig::{completion::Prompt, providers::openai};

#[tokio::main]
async fn main() {
    // Create OpenAI client and agent.
    // This requires the `OPENAI_API_KEY` environment variable to be set.
    let openai_client = openai::Client::from_env();

    let gpt4 = openai_client.agent("gpt-4").build();

    // Prompt the model and print its response
    let response = gpt4
        .prompt("Who are you?")
        .await
        .expect("Failed to prompt GPT-4");

    println!("GPT-4: {response}");
}

Note: using #[tokio::main] requires you enable tokio’s macros and rt-multi-thread features or just full to enable all features (cargo add tokio --features macros,rt-multi-thread).

§Core concepts

§Completion and embedding models

Rig provides a consistent API for working with LLMs and embeddings. Specifically, each provider (e.g. OpenAI, Cohere) has a Client struct that can be used to initialize completion and embedding models. These models implement the CompletionModel and EmbeddingModel traits respectively, which provide a common, low-level interface for creating completion and embedding requests and executing them.

§Agents

Rig also provides high-level abstractions over LLMs in the form of the Agent type.

The Agent type can be used to create anything from simple agents that use vanilla models to full blown RAG systems that can be used to answer questions using a knowledge base.

§Vector stores and indexes

Rig provides a common interface for working with vector stores and indexes. Specifically, the library provides the VectorStoreIndex trait, which can be implemented to define vector stores and indices respectively. Those can then be used as the knowledge base for a RAG enabled Agent, or as a source of context documents in a custom architecture that use multiple LLMs or agents.

§Integrations

§Model Providers

Rig natively supports the following completion and embedding model provider integrations:

  • OpenAI
  • Cohere
  • Anthropic
  • Perplexity
  • Google Gemini
  • xAI
  • DeepSeek

You can also implement your own model provider integration by defining types that implement the CompletionModel and EmbeddingModel traits.

§Vector Stores

Rig currently supports the following vector store integrations via companion crates:

  • rig-mongodb: Vector store implementation for MongoDB
  • rig-lancedb: Vector store implementation for LanceDB
  • rig-neo4j: Vector store implementation for Neo4j
  • rig-qdrant: Vector store implementation for Qdrant

You can also implement your own vector store integration by defining types that implement the VectorStoreIndex trait.

Re-exports§

pub use completion::message;
pub use embeddings::Embed;
pub use one_or_many::EmptyListError;
pub use one_or_many::OneOrMany;

Modules§

agent
This module contains the implementation of the Agent struct and its builder.
cli_chatbot
completion
embeddings
This module provides functionality for working with embeddings. Embeddings are numerical representations of documents or other objects, typically used in natural language processing (NLP) tasks such as text classification, information retrieval, and document similarity.
extractor
This module provides high-level abstractions for extracting structured data from text using LLMs.
loaders
This module provides utility structs for loading and preprocessing files.
one_or_many
pipeline
This module defines a flexible pipeline API for defining a sequence of operations that may or may not use AI components (e.g.: semantic search, LLMs prompting, etc).
providers
This module contains clients for the different LLM providers that Rig supports.
streaming
This module provides functionality for working with streaming completion models. It provides traits and types for generating streaming completion requests and handling streaming completion responses.
tool
Module defining tool related structs and traits.
transcription
This module provides functionality for working with audio transcription models. It provides traits, structs, and enums for generating audio transcription requests, handling transcription responses, and defining transcription models.
vector_store

Macros§

conditional
Creates an Op that conditionally dispatches to one of multiple sub-ops based on the variant of the input enum.
parallel
parallel_internal
parallel_op
try_conditional
Creates a TryOp that conditionally dispatches to one of multiple sub-ops based on the variant of the input enum, returning a Result.
try_parallel
try_parallel_internal
tuple_pattern