🦜️🔗LangChain AI Rust

⚡ Building applications with LLMs through composability, with Rust! ⚡

🤔 What is this?

This is the Rust language implementation of LangChain, providing a powerful and type-safe way to build LLM applications in Rust.

✨ Key Features

• 🚀 Multiple LLM Providers: Support for OpenAI, Azure OpenAI, Anthropic Claude, MistralAI, Google Gemini, AWS Bedrock, HuggingFace, Alibaba Qwen, DeepSeek, and Ollama
• 🔗 Chains: LLM chains, conversational chains, sequential chains, Q&A chains, SQL chains, and more
• 🤖 Agents: Chat agents with tools, multi-agent systems (router, subagents, skills, handoffs)
• 📚 RAG: Agentic RAG, Hybrid RAG, and two-step RAG implementations
• 🧠 Memory: Simple memory, conversational memory, and long-term memory with metadata
• 🛠️ Tools: Search tools, command line, Wolfram Alpha, text-to-speech, and more
• 📄 Document Loaders: PDF, HTML, CSV, Git commits, source code, and more
• 🗄️ Vector Stores: PostgreSQL (pgvector), Qdrant, SQLite (VSS/Vec), SurrealDB, OpenSearch, In-Memory, Chroma, FAISS (hnsw_rs), MongoDB Atlas, Pinecone, Weaviate
• 🎯 Embeddings: OpenAI, Azure OpenAI, Ollama, FastEmbed, MistralAI
• 🔧 Middleware: Logging, PII detection, content filtering, rate limiting, retry, and custom middleware
• 🎨 Structured Output: JSON schema validation and structured response generation
• ⚙️ Runtime Context: Dynamic prompts, typed context, and runtime-aware middleware
• 📊 LangGraph: State graphs, streaming, persistence (SQLite/memory), interrupts, subgraphs, and time-travel debugging
• 🤖 Deep Agent: Planning (write_todos), filesystem tools (ls, read_file, write_file, edit_file), skills, long-term memory, and human-in-the-loop

📦 Installation

This library heavily relies on serde_json for its operation.

Step 1: Add serde_json

First, ensure serde_json is added to your Rust project.

cargo add serde_json

Step 2: Add langchain-ai-rust

Then, you can add langchain-ai-rust to your Rust project.

Simple install

cargo add langchain-ai-rust

With Vector Stores

PostgreSQL (pgvector)

cargo add langchain-ai-rust --features postgres

Qdrant

cargo add langchain-ai-rust --features qdrant

SQLite (VSS)

Download additional sqlite_vss libraries from https://github.com/asg017/sqlite-vss

cargo add langchain-ai-rust --features sqlite-vss

SQLite (Vec)

Download additional sqlite_vec libraries from https://github.com/asg017/sqlite-vec

cargo add langchain-ai-rust --features sqlite-vec

SurrealDB

cargo add langchain-ai-rust --features surrealdb

OpenSearch

cargo add langchain-ai-rust --features opensearch

In-Memory

cargo add langchain-ai-rust --features in-memory

Chroma

cargo add langchain-ai-rust --features chroma

FAISS (hnsw_rs)

cargo add langchain-ai-rust --features faiss

MongoDB Atlas Vector Search

cargo add langchain-ai-rust --features mongodb

Pinecone

cargo add langchain-ai-rust --features pinecone

Weaviate

cargo add langchain-ai-rust --features weaviate

With LLM Providers

Ollama

cargo add langchain-ai-rust --features ollama

MistralAI

cargo add langchain-ai-rust --features mistralai

Google Gemini

cargo add langchain-ai-rust --features gemini

AWS Bedrock

cargo add langchain-ai-rust --features bedrock

With Document Loaders

PDF (pdf-extract)

cargo add langchain-ai-rust --features pdf-extract

PDF (lopdf)

cargo add langchain-ai-rust --features lopdf

HTML to Markdown

cargo add langchain-ai-rust --features html-to-markdown

With Code Parsing

Tree-sitter (for source code parsing, requires 0.26+)

cargo add langchain-ai-rust --features tree-sitter

With FastEmbed (Local Embeddings)

cargo add langchain-ai-rust --features fastembed

🚀 Quick Start

Simple LLM Invocation

use langchain_ai_rust::llm::openai::{OpenAI, OpenAIModel};

#[tokio::main]
async fn main() {
    let llm = OpenAI::default().with_model(OpenAIModel::Gpt4oMini.to_string());
    let response = llm.invoke("What is Rust?").await.unwrap();
    println!("{}", response);
}

Using init_chat_model (Recommended)

The init_chat_model function provides a unified interface to initialize any supported LLM:

use langchain_ai_rust::language_models::init_chat_model;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize any supported model
    let model = init_chat_model("gpt-4o-mini", None, None, None, None, None, None, None).await?;

    let response = model.invoke("Hello, world!").await?;
    println!("{}", response);

    Ok(())
}

Supported model formats:

• gpt-4o-mini, gpt-4o, gpt-4-turbo (OpenAI)
• claude-3-5-sonnet-20241022 (Anthropic)
• mistralai/mistral-large-latest (MistralAI)
• gemini-1.5-pro (Google Gemini)
• anthropic.claude-3-5-sonnet-20241022-v2:0 (AWS Bedrock)
• meta-llama/Llama-3.1-8B-Instruct (HuggingFace)
• qwen-plus (Alibaba Qwen)
• deepseek-chat (DeepSeek)
• llama3 (Ollama)

Conversational Chain

use langchain_ai_rust::{
    chain::{Chain, LLMChainBuilder},
    fmt_message, fmt_placeholder, fmt_template,
    llm::openai::{OpenAI, OpenAIModel},
    message_formatter,
    prompt::HumanMessagePromptTemplate,
    prompt_args,
    schemas::messages::Message,
    template_fstring,
};

#[tokio::main]
async fn main() {
    let open_ai = OpenAI::default().with_model(OpenAIModel::Gpt4oMini.to_string());

    let prompt = message_formatter![
        fmt_message!(Message::new_system_message(
            "You are a helpful assistant."
        )),
        fmt_placeholder!("history"),
        fmt_template!(HumanMessagePromptTemplate::new(template_fstring!(
            "{input}", "input"
        ))),
    ];

    let chain = LLMChainBuilder::new()
        .prompt(prompt)
        .llm(open_ai)
        .build()
        .unwrap();

    match chain
        .invoke(prompt_args! {
            "input" => "What is Rust?",
            "history" => vec![
                Message::new_human_message("Hello"),
                Message::new_ai_message("Hi there!"),
            ],
        })
        .await
    {
        Ok(result) => println!("Result: {:?}", result),
        Err(e) => panic!("Error: {:?}", e),
    }
}

Creating an Agent with Tools

use std::sync::Arc;
use langchain_ai_rust::{
    agent::create_agent,
    schemas::messages::Message,
    tools::CommandExecutor,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let command_executor = Arc::new(CommandExecutor::default());

    let agent = create_agent(
        "gpt-4o-mini",
        &[command_executor],
        Some("You are a helpful assistant that can execute commands."),
        None,
    )?;

    let result = agent
        .invoke_messages(vec![Message::new_human_message(
            "What files are in the current directory?",
        )])
        .await?;

    println!("{}", result);
    Ok(())
}

LangGraph (Hello World)

Build a state graph with MessagesState, add a node with function_node, connect START to the node and the node to END, then compile and invoke:

use langchain_ai_rust::langgraph::{function_node, MessagesState, StateGraph, END, START};
use langchain_ai_rust::schemas::messages::Message;

let mock_llm = function_node("mock_llm", |_state: &MessagesState| async move {
    use std::collections::HashMap;
    let mut update = HashMap::new();
    update.insert(
        "messages".to_string(),
        serde_json::to_value(vec![Message::new_ai_message("hello world")])?,
    );
    Ok(update)
});

let mut graph = StateGraph::<MessagesState>::new();
graph.add_node("mock_llm", mock_llm)?;
graph.add_edge(START, "mock_llm");
graph.add_edge("mock_llm", END);

let compiled = graph.compile()?;
let initial_state = MessagesState::with_messages(vec![Message::new_human_message("hi!")]);
let final_state = compiled.invoke(initial_state).await?;

See LangGraph Hello World and LangGraph Streaming for more.

Deep Agent (Basic)

Use create_deep_agent with planning and filesystem enabled; the agent gets a workspace and built-in tools (write_todos, ls, read_file, write_file, edit_file):

use langchain_ai_rust::{
    agent::{create_deep_agent, DeepAgentConfig},
    chain::Chain,
    prompt_args,
    schemas::messages::Message,
};

let workspace = std::env::temp_dir().join("my_agent_workspace");
std::fs::create_dir_all(&workspace)?;

let config = DeepAgentConfig::new()
    .with_planning(true)
    .with_filesystem(true)
    .with_workspace_root(workspace);

let agent = create_deep_agent(
    "gpt-4o-mini",
    &[],
    Some("You are a helpful assistant with planning and file tools."),
    config,
)?;

let result = agent
    .invoke(prompt_args! {
        "messages" => vec![Message::new_human_message("List files in the workspace.")]
    })
    .await?;

See Deep Agent Basic and Deep Agent Customization for more.

📚 Current Features

LLMs

• OpenAI
• Azure OpenAI
• Anthropic Claude
• MistralAI
• Google Gemini
• AWS Bedrock
• HuggingFace
• Alibaba Qwen
• DeepSeek
• Ollama
• Unified Model Initialization

Embeddings

• OpenAI
• Azure OpenAI
• Ollama
• Local FastEmbed
• MistralAI

Vector Stores

• PostgreSQL (pgvector)
• Qdrant
• SQLite VSS
• SQLite Vec
• SurrealDB
• OpenSearch
• In-Memory
• Chroma
• FAISS
• MongoDB Atlas
• Pinecone
• Weaviate

Chains

• LLM Chain
• Conversational Chain
• Conversational Retriever Simple
• Conversational Retriever With Vector Store
• Sequential Chain
• Q&A Chain
• SQL Chain
• Streaming Chain

Agents

• Simple Agent
• Chat Agent with Tools
• OpenAI Compatible Tools Agent
• Multi-Agent Router
• Multi-Agent Subagents
• Multi-Agent Skills
• Multi-Agent Handoffs

LangGraph

• Hello World
• Streaming
• Persistence Basic, Persistence SQLite, Persistence Replay
• Interrupts, Interrupts Approval, Interrupts Review
• Subgraph Shared State, Subgraph Streaming
• Memory Store, Memory Basic
• Agent Workflow, Parallel Execution, Time Travel, Task Example

Deep Agent

• Basic (planning + filesystem)
• Customization
• Skills
• Planning
• Filesystem
• Human-in-the-Loop
• Long-term Memory
• With Task Tool

Text Splitters

Text Structure-Based

• RecursiveCharacterTextSplitter - Recommended default, splits recursively by separators
• CharacterTextSplitter - Simple character-based splitting with single separator
• PlainTextSplitter - Basic text splitting
• TokenSplitter - Token-based splitting (Tiktoken)

Document Structure-Based

• MarkdownSplitter - Split Markdown by structure
• HTMLSplitter - Split HTML by tags
• JsonSplitter - Split JSON by objects/arrays
• CodeSplitter - Split code by syntax tree (tree-sitter 0.26+, requires tree-sitter feature)

RAG (Retrieval-Augmented Generation)

• Agentic RAG - Agent decides when to retrieve
• Hybrid RAG - Combines multiple retrieval strategies
• Two-Step RAG - Two-stage retrieval process

Retrievers

External Index Retrievers

• Wikipedia Retriever - Retrieve Wikipedia articles
• Arxiv Retriever - Retrieve academic papers from arXiv
• Tavily Search API Retriever - Real-time web search

Algorithm-Based Retrievers

• BM25 Retriever - BM25 algorithm for text retrieval
• TF-IDF Retriever - TF-IDF based retrieval
• SVM Retriever - Support Vector Machine based retrieval

Rerankers

• Cohere Reranker - Rerank using Cohere API
• FlashRank Reranker - Local ONNX model reranking
• Contextual AI Reranker - Contextual AI API reranking

Hybrid Retrievers

• Merger Retriever - Combine multiple retrievers
• Ensemble Retriever - Voting mechanism from multiple retrievers

Query Enhancement Retrievers

• RePhrase Query Retriever - LLM-based query rephrasing
• Multi Query Retriever - Generate multiple query variations

Document Compression Retrievers

• Embeddings Redundant Filter - Filter redundant documents by similarity

Tools

• Serpapi/Google Search
• DuckDuckGo Search
• Wolfram Alpha
• Command Line Executor
• Text-to-Speech
• Speech-to-Text
• Advanced Tools

Middleware

• Logging Middleware
• PII Detection
• Content Filtering
• Custom Middleware
• Runtime-Aware Middleware
• Dynamic Prompt Middleware

Memory

• Simple Memory
• Conversational Memory
• Long-Term Memory (Basic)
• Long-Term Memory (Search)
• Long-Term Memory (Tool)

Runtime & Context

• Typed Context
• Dynamic Tools
• Dynamic Prompts
• Message Injection
• Complete Context Engineering

Structured Output

• Structured Output
• Structured Output Provider

Advanced Features

• Configurable Models
• Invocation Config
• Semantic Routing
• Dynamic Semantic Routing
• Vision LLM Chain
• Tool Runtime

Document Loaders

Common File Types

• PDF (pdf-extract or lopdf)
• HTML
• HTML to Markdown
• CSV
• TSV (Tab-Separated Values)
• JSON (including JSONL)
• Markdown
• TOML (with toml feature)
• YAML (with yaml feature)
• XML (with xml feature)

Office Documents

• Excel (.xlsx, .xls) (with excel feature)
• Word, PowerPoint, and more via PandocLoader

Web Loaders

• WebBaseLoader - Load content from URLs
• RecursiveURLLoader - Recursively crawl websites
• SitemapLoader - Load all URLs from sitemap.xml (with xml feature)

Cloud Storage

• AWS S3 (with aws-s3 feature)

Productivity Tools

• GitHub (with github feature)
• Git Commits (with git feature)

Other

• Source Code (with tree-sitter feature)
• Pandoc (various formats: docx, epub, html, ipynb, markdown, etc.)

See the examples directory for complete examples of each feature.

🔧 Configuration

Environment Variables

For OpenAI:

export OPENAI_API_KEY="your-api-key"

For Anthropic:

export ANTHROPIC_API_KEY="your-api-key"

For MistralAI:

export MISTRAL_API_KEY="your-api-key"

For Google Gemini:

export GOOGLE_API_KEY="your-api-key"

For AWS Bedrock:

export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_REGION="us-east-1"

📖 Documentation

• Examples Directory
• API Documentation

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• LangChain - The original Python implementation
• All contributors and users of this library

🔗 Links

• Crates.io
• Discord
• GitHub Repository