langchainrust

A LangChain-inspired Rust framework for building LLM applications.

What it solves: Build Agents, RAG, BM25 keyword search, Hybrid retrieval, LangGraph workflows, MCP tools, Guardrails, multi-agent Handoffs — all in pure Rust.

Core Features

Full documentation: Usage Guide | API Docs

Architecture

┌─────────────────────────────────────────────────────────────┐
│                      langchainrust                           │
├─────────────────────────────────────────────────────────────┤
│  LLM Layer                                                   │
│  ├── OpenAIChat / OllamaChat                                 │
│  ├── DeepSeek / Moonshot / Zhipu / Qwen (OpenAI compatible) │
│  ├── AnthropicChat (Claude API) / GeminiChat                 │
│  ├── Function Calling (bind_tools) / Streaming (stream_chat)│
│  ├── Multimodal Vision (ImageContent + human_with_image)    │
│  ├── OpenAI Assistants API (with requires_action dispatch)   │
│  ├── OpenAI Responses API (web_search/file_search/code/...)  │
│  ├── Anthropic Extended Thinking (with_thinking)             │
│  ├── RouterLLM (5 strategies + Fallback)                     │
│  ├── BatchClient (OpenAI/Anthropic batch inference)          │
│  └── with_structured_output (StructuredOutputExt trait)      │
├─────────────────────────────────────────────────────────────┤
│  Embeddings Layer                                            │
│  ├── OpenAIEmbeddings / DeepSeekEmbeddings                   │
│  ├── QwenEmbeddings / MockEmbeddings                         │
│  └── LocalEmbeddings (ort ONNX Runtime, feature gate)       │
├─────────────────────────────────────────────────────────────┤
│  Agent Layer                                                 │
│  ├── ReActAgent / FunctionCallingAgent                      │
│  ├── Plan-Execute Agent (plan -> execute -> replan)         │
│  ├── Handoffs (multi-agent handoff) / Streaming FC          │
│  ├── GuardedAgent (Guardrails safety)                       │
│  ├── DeepResearchAgent (multi-round research + citations)   │
│  ├── AgentExecutor (parallel tool calls, bounded concurrency)│
│  ├── Supervisor (LLM routes tasks to sub-agents, FINISH)    │
│  ├── A2A Server/Client (Agent-to-Agent protocol)            │
│  └── LangGraph (StateGraph, Subgraph, Parallel, dynamic     │
│  │              interrupt/resume, history + fork time-travel)│
├─────────────────────────────────────────────────────────────┤
│  MCP Layer                                                   │
│  ├── StatelessMcpClient (HTTP POST) -> MCPToolAdapter        │
│  ├── MCPServer (expose BaseTool to host)                     │
│  └── Full 6 primitives (resources/prompts/completion/...)    │
├─────────────────────────────────────────────────────────────┤
│  Retrieval Layer                                             │
│  ├── RAG (TextSplitter, SemanticSplitter, VectorStore)      │
│  ├── BM25 (Keyword Search, AutoMerging)                     │
│  ├── Hybrid (BM25 + Vector, RRF/Weighted fusion, MMR)       │
│  ├── Neural Rerank (Cohere/Jina) / Sentence-Window / Parent │
│  ├── HyDE / MultiQuery / Reranking / late chunking          │
│  ├── CorrectiveRAG (grade + rewrite + hallucination detect) │
│  ├── AdaptiveRAG (LLM-routed retrieval strategy)            │
│  ├── GraphRAG (knowledge graph + community detection)       │
│  └── Loaders (Text/JSON/MD/PDF/CSV/HTML/Docx/Web/Sitemap)  │
├─────────────────────────────────────────────────────────────┤
│  Storage Layer                                               │
│  ├── Vector DB (InMemory, Qdrant, MongoDB, ChromaDB,        │
│  │              Redis, SQLite, PGVector, Pinecone, File)    │
│  └── Sessions (EventSessionManager + EventStore)            │
├─────────────────────────────────────────────────────────────┤
│  Utility Layer                                               │
│  ├── Memory (Buffer, Window, Summary, SummaryBuffer, Vector,│
│  │           ContextWindow[Truncate+Summarize])             │
│  ├── Chains (LLMChain, SequentialChain, RetrievalQA, ...)   │
│  │         + Chain streaming (per-token output)              │
│  ├── Prompts (PromptTemplate, ChatPromptTemplate, FewShot)  │
│  ├── Tools (Calculator, DateTime, URLFetch, HTTP/File/SQL,  │
│  │          ComputerUseTool, CodeSandbox)                    │
│  ├── Output Parsers                                         │
│  ├── Token Counter (Tiktoken + Cost Tracking)               │
│  ├── LLM Cache                                              │
│  ├── Evaluation (10+ evaluators, RAGAS, golden/compare gate)│
│  ├── Tracing (Tracer + SpanGuard, InMemory/Console/OTel)   │
│  └── Callbacks (LangSmith, StdOut, FileHandler, Otel)       │
└─────────────────────────────────────────────────────────────┘

Installation

[dependencies]
langchainrust = "0.24.0"
tokio = { version = "1.0", features = ["full"] }

# Optional features
langchainrust = { version = "0.24.0", features = ["mongodb-persistence"] }  # MongoDB storage
langchainrust = { version = "0.24.0", features = ["qdrant-integration"] }    # Qdrant vector DB
langchainrust = { version = "0.24.0", features = ["redis-storage"] }         # Redis storage
langchainrust = { version = "0.24.0", features = ["sqlite-storage"] }        # SQLite storage (+ SQLTool)
langchainrust = { version = "0.24.0", features = ["pgvector-storage"] }      # PGVector (requires user-configured sqlx/pgvector deps)
langchainrust = { version = "0.24.0", features = ["local-embeddings"] }      # Local ONNX embeddings (requires ort)
langchainrust = { version = "0.24.0", features = ["opentelemetry"] }         # OpenTelemetry tracing
# PineconeStore / FileVectorStore require no feature flag, available by default

Rust version requirements (MSRV)

• Default features: Rust 1.85+ — this is the workspace MSRV, enforced in CI and used for MSRV-aware dependency resolution (resolver = "3").
• The local-model features local-embeddings (ONNX via ort), fastembed, and local-candle (candle) pull crates that require a newer toolchain (currently Rust 1.88+). Cargo cannot declare an MSRV per feature, so this layering is documented rather than compile-enforced: use these three features only with a recent stable Rust. docs.rs builds them on the latest nightly automatically.

Quick Start

use langchainrust::{OpenAIChat, OpenAIConfig, BaseChatModel};
use langchainrust::schema::Message;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = OpenAIConfig {
        api_key: std::env::var("OPENAI_API_KEY")?,
        base_url: "https://api.openai.com/v1".to_string(),
        model: "gpt-3.5-turbo".to_string(),
        ..Default::default()
    };

    let llm = OpenAIChat::new(config);

    let response = llm.chat(vec![
        Message::system("You are a helpful assistant."),
        Message::human("What is Rust?"),
    ], None).await?;

    println!("{}", response.content);
    Ok(())
}

Multi-Provider Support

use langchainrust::{
    DeepSeekChat, MoonshotChat, ZhipuChat, QwenChat,
    AnthropicChat, OllamaChat,
};

let deepseek = DeepSeekChat::from_env();
let moonshot = MoonshotChat::with_model("moonshot-v1-128k");
let claude = AnthropicChat::from_env();
let ollama = OllamaChat::new("llama3.2");

BM25 Keyword Search

use langchainrust::{BM25Retriever, Document};

let mut retriever = BM25Retriever::new();

retriever.add_documents_sync(vec![
    Document::new("Rust is a systems programming language"),
    Document::new("Python is a scripting language"),
]);

let results = retriever.search("systems programming", 3);

for result in results {
    println!("Document: {}", result.document.content);
    println!("Score: {}", result.score);
}

More examples in Usage Guide.

Examples

The examples/ directory provides 42 runnable examples covering core functionality:

Examples requiring API keys read from environment variables:

export OPENAI_API_KEY="your-key"
cargo run --example basic_chat

Examples without API keys (BM25 / LangGraph / Memory / Loader) can run directly — great for quick exploration.

Documentation

Testing

cargo test

Contributing

Contributions welcome! See CONTRIBUTING.md.

License

MIT or Apache-2.0, at your option.