Crate rig

Rig is a Rust library for building LLM-powered applications that focuses on ergonomics and modularity.

Table of contents

• High-level features
• Simple Example
• Core Concepts
• Integrations

High-level features

• Full support for LLM completion and embedding workflows
• Simple but powerful common abstractions over LLM providers (e.g. OpenAI, Cohere) and vector stores (e.g. MongoDB, in-memory)
• Integrate LLMs in your app with minimal boilerplate

Simple example:

use rig::{completion::Prompt, providers::openai};

#[tokio::main]
async fn main() {
    // Create OpenAI client and agent.
    // This requires the `OPENAI_API_KEY` environment variable to be set.
    let openai_client = openai::Client::from_env();

    let gpt4 = openai_client.agent("gpt-4").build();

    // Prompt the model and print its response
    let response = gpt4
        .prompt("Who are you?")
        .await
        .expect("Failed to prompt GPT-4");

    println!("GPT-4: {response}");
}

Note: using #[tokio::main] requires you enable tokio’s macros and rt-multi-thread features or just full to enable all features (cargo add tokio --features macros,rt-multi-thread).

Core concepts

Completion and embedding models

Rig provides a consistent API for working with LLMs and embeddings. Specifically, each provider (e.g. OpenAI, Cohere) has a Client struct that can be used to initialize completion and embedding models. These models implement the CompletionModel and EmbeddingModel traits respectively, which provide a common, low-level interface for creating completion and embedding requests and executing them.

Agents

Rig also provides high-level abstractions over LLMs in the form of the Agent type.

The Agent type can be used to create anything from simple agents that use vanilla models to full blown RAG systems that can be used to answer questions using a knowledge base.

Vector stores and indexes

Rig provides a common interface for working with vector stores and indexes. Specifically, the library provides the VectorStoreIndex trait, which can be implemented to define vector stores and indices respectively. Those can then be used as the knowledge base for a RAG enabled Agent, or as a source of context documents in a custom architecture that use multiple LLMs or agents.

Integrations

Model Providers

Rig natively supports the following completion and embedding model provider integrations:

• OpenAI
• Cohere
• Anthropic
• Perplexity
• Gemini

You can also implement your own model provider integration by defining types that implement the CompletionModel and EmbeddingModel traits.

Vector Stores

Rig currently supports the following vector store integrations via companion crates:

• rig-mongodb: Vector store implementation for MongoDB
• rig-lancedb: Vector store implementation for LanceDB
• rig-neo4j: Vector store implementation for Neo4j
• rig-qdrant: Vector store implementation for Qdrant

You can also implement your own vector store integration by defining types that implement the VectorStoreIndex trait.

Re-exports

• pub use embeddings::Embed;
• pub use one_or_many::EmptyListError;
• pub use one_or_many::OneOrMany;

Modules

• agent
  This module contains the implementation of the Agent struct and its builder.
• cli_chatbot
• completion
  This module provides functionality for working with completion models. It provides traits, structs, and enums for generating completion requests, handling completion responses, and defining completion models.
• embeddings
  This module provides functionality for working with embeddings. Embeddings are numerical representations of documents or other objects, typically used in natural language processing (NLP) tasks such as text classification, information retrieval, and document similarity.
• extractor
  This module provides high-level abstractions for extracting structured data from text using LLMs.
• loaders
  This module provides utility structs for loading and preprocessing files.
• one_or_many
• providers
  This module contains clients for the different LLM providers that Rig supports.
• tool
  Module defining tool related structs and traits.
• vector_store