Crate rig

Rig is a Rust library for building LLM-powered applications that focuses on ergonomics and modularity.

Table of contents

• High-level features
• Simple Example
• Core Concepts
• Integrations

High-level features

• Full support for LLM completion and embedding workflows
• Simple but powerful common abstractions over LLM providers (e.g. OpenAI, Cohere) and vector stores (e.g. MongoDB, in-memory)
• Integrate LLMs in your app with minimal boilerplate

Simple example:

use rig::{completion::Prompt, providers::openai};

#[tokio::main]
async fn main() {
    // Create OpenAI client and model.
    // This requires the `OPENAI_API_KEY` environment variable to be set.
    let openai_client = openai::Client::from_env();

    let gpt4 = openai_client.model("gpt-4").build();

    // Prompt the model and print its response
    let response = gpt4
        .prompt("Who are you?")
        .await
        .expect("Failed to prompt GPT-4");

    println!("GPT-4: {response}");
}

Note: using #[tokio::main] requires you enable tokio’s macros and rt-multi-thread features or just full to enable all features (cargo add tokio --features macros,rt-multi-thread).

Core concepts

Completion and embedding models

Rig provides a consistent API for working with LLMs and embeddings. Specifically, each provider (e.g. OpenAI, Cohere) has a Client struct that can be used to create completion and embedding models. These models implement the CompletionModel and EmbeddingModel traits respectively, which provide a common, low-level interface for creating completion and embedding requests and executing them.

Models, Agents and RagAgents

Rig provides high-level abstractions over LLMs in the form of Model, Agent and RagAgent structs.

These structs range from simple models that can be prompted directly to agents that have a system prompt to full blown RAG systems that can be used to answer questions using a knowledgebase. Here is a quick summary of each:

• Model: A simple LLM model that can be prompted directly. This structs acts as a thin wrapper around a completion model (i.e.: a struct implementing the CompletionModel trait).
• Agent: An LLM model combined with a preamble (i.e.: system prompt) and a static set of context documents and tools.
• RagAgent: A RAG system that can be used to answer questions using a knowledgebase containing both context documents and tools.

Vector stores and indexes

Rig provides a common interface for working with vector stores and indexes. Specifically, the library provides the VectorStore and VectorStoreIndex traits, which can be implemented to define vector stores and indices respectively. Those can then be used as the knowledgebase for a RagAgent, or as a source of context documents in a custom architecture that use multiple LLMs or agents.

Integrations

Rig natively supports the following completion and embedding model providers:

• OpenAI
• Cohere

Rig currently has the following integration companion crates:

• rig-mongodb: Vector store implementation for MongoDB

Modules

• agent
  This module contains the implementation of the Agent struct and its builder.
• cli_chatbot
• completion
  This module provides functionality for working with completion models. It provides traits, structs, and enums for generating completion requests, handling completion responses, and defining completion models.
• embeddings
  This module provides functionality for working with embeddings and embedding models. Embeddings are numerical representations of documents or other objects, typically used in natural language processing (NLP) tasks such as text classification, information retrieval, and document similarity.
• extractor
  This module provides high-level abstractions for extracting structured data from text using LLMs.
• json_utils
• model
  This module contains the implementation of the Model struct and its builder.
• providers
  This module contains clients for the different LLM providers that Rig supports.
• rag
  This module contains the implementation of the RagAgent struct and its builder.
• tool
• vector_store