# langchainrust
[](https://www.rust-lang.org/)
[](LICENSE)
[](https://crates.io/crates/langchainrust)
[](https://docs.rs/langchainrust)
[](https://github.com/atliliw/langchainrust/actions/workflows/ci.yml)
[](https://crates.io/crates/langchainrust)
A LangChain-inspired Rust framework for building LLM applications.
**What it solves**: Build Agents, RAG, BM25 keyword search, Hybrid retrieval, LangGraph workflows, MCP tools, Guardrails, multi-agent Handoffs — all in pure Rust.
---
## Core Features
| **LLM** | OpenAI / Ollama / DeepSeek / Moonshot / Zhipu / Qwen / Anthropic Claude / Gemini + Multimodal Vision + Assistants API (with requires_action tool dispatch) |
| **Embeddings** | OpenAI / DeepSeek / Qwen / Local (ort ONNX Runtime, feature gate) / Mock |
| **Agents** | ReActAgent / FunctionCallingAgent / Plan-Execute / Handoffs (multi-agent handoff) / Streaming Function Calling / **Supervisor dynamic sub-agent routing (v0.24.0)** / parallel tool calls (bounded concurrency) |
| **A2A** | Agent-to-Agent protocol, AgentCard/Task/Message + Server (with task persistence) + Client |
| **MCP** | Model Context Protocol Client + Server (stdio + Streamable HTTP, stateless track, OAuth 2.1), full 6 primitives, MCP tool adapter to BaseTool |
| **Memory** | Buffer / Window / Summary / SummaryBuffer / Persistent / VectorStore (semantic retrieval) / ContextWindow (Truncate + Summarize) |
| **Sessions** | Multi-turn event-sourced lifecycle (`EventSessionManager` + `EventStore`, fork-from-any-point), pluggable storage |
| **Chains** | LLMChain / SequentialChain / ConversationChain / RouterChain / RetrievalQA / ConversationRetrieval / Stuff / Refine / MapReduce + Chain streaming |
| **RAG** | Document splitting (including SemanticSplitter), vector store, semantic retrieval, MultiQuery, HyDE, Reranking, query_with_sources (citation tracing), late chunking (dual-leg hybrid injection, v0.24.0) |
| **Neural rerank (v0.24.0)** | `AsyncReranker` trait with hosted Cohere / Jina cross-encoders and the `rerank_async` helper (out-of-order index mapping, proxy bypass) |
| **Small-to-big (v0.24.0)** | `SentenceWindowRetriever` (±N-sentence windows) and `ParentDocumentRetriever` (leaf hits return the whole parent document) |
| **Structured Output** | with_structured_output, StructuredOutputExt trait + JsonOutputParser fallback, Streaming Structured Output |
| **BM25** | Keyword search, Chinese/English tokenization, AutoMerging, Chunked |
| **Hybrid** | BM25 + Vector hybrid retrieval, RRF / weighted linear fusion, MMR diversity re-ranking, Unified index |
| **LangGraph** | Graph workflows; static and **dynamic in-node interrupt/resume** human-in-the-loop; **state history & fork time travel**; Subgraph; Parallel; Checkpointer (memory/file/SQLite/Postgres/Redis) |
| **Guardrails** | Input/output safety guardrails, SensitiveInfo / ForbiddenWords / MaxLength, GuardedAgent |
| **Token Counter** | Tiktoken counting + TokenTrackingLLM usage statistics + ModelPricing cost estimation |
| **Output Parsers** | StrOutputParser, JsonOutputParser, CommaSeparatedList, Structured, Typed |
| **Tools** | Calculator / DateTime / Math / URLFetch / Wikipedia / WebSearch / PythonREPL / HTTPTool / FileTool (sandbox) / SQLTool (read-only) / ComputerUseTool |
| **Vector DB** | InMemory / Qdrant / MongoDB / ChromaDB / Redis / SQLite / PGVector / Pinecone / FileVectorStore |
| **Document Loaders** | Text / JSON / Markdown / PDF / CSV / HTML + WebScraper / Sitemap / Docx |
| **Cache** | LLMCache with TTL support |
| **Prompts** | PromptTemplate / ChatPromptTemplate / FewShotPromptTemplate |
| **Callbacks** | StdOut / LangSmith / FileHandler / OpenTelemetry |
| **Evaluation** | ExactMatch / StringDistance / EmbeddingSimilarity / LLMAsJudge / PairwiseJudge / ContainsKeyword / RegexMatch / LengthCheck / Bleu / Faithfulness + RAGAS metrics; v0.24 adds trace→golden dataset bridging (lc-testkit) and `compare_reports` regression gating |
| **Advanced RAG** | CorrectiveRAG (self-correcting) / AdaptiveRAG (adaptive retrieval) / GraphRAG (knowledge graph) |
| **Model Routing** | RouterLLM with 6 strategies (Fallback / RoundRobin / LeastLatency / LatencyWeighted / LowestCost / InputDirected) |
| **Deep Research** | Multi-round deep research agent with sub-topic decomposition, parallel search, deduplication, and citation reporting |
| **Code Interpreter** | LocalSandbox (subprocess + timeout) |
| **Batch API** | BatchClient for OpenAI/Anthropic batch inference, 50% cost reduction |
| **Tracing** | Tracer + SpanGuard (RAII), InMemory / Console / OTel backends, parent-child span tree |
Full documentation: [Usage Guide](https://github.com/atliliw/langchainrust/blob/main/docs/USAGE_EN.md) | [API Docs](https://docs.rs/langchainrust)
---
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ langchainrust │
├─────────────────────────────────────────────────────────────┤
│ LLM Layer │
│ ├── OpenAIChat / OllamaChat │
│ ├── DeepSeek / Moonshot / Zhipu / Qwen (OpenAI compatible) │
│ ├── AnthropicChat (Claude API) / GeminiChat │
│ ├── Function Calling (bind_tools) / Streaming (stream_chat)│
│ ├── Multimodal Vision (ImageContent + human_with_image) │
│ ├── OpenAI Assistants API (with requires_action dispatch) │
│ ├── OpenAI Responses API (web_search/file_search/code/...) │
│ ├── Anthropic Extended Thinking (with_thinking) │
│ ├── RouterLLM (5 strategies + Fallback) │
│ ├── BatchClient (OpenAI/Anthropic batch inference) │
│ └── with_structured_output (StructuredOutputExt trait) │
├─────────────────────────────────────────────────────────────┤
│ Embeddings Layer │
│ ├── OpenAIEmbeddings / DeepSeekEmbeddings │
│ ├── QwenEmbeddings / MockEmbeddings │
│ └── LocalEmbeddings (ort ONNX Runtime, feature gate) │
├─────────────────────────────────────────────────────────────┤
│ Agent Layer │
│ ├── ReActAgent / FunctionCallingAgent │
│ ├── Plan-Execute Agent (plan -> execute -> replan) │
│ ├── Handoffs (multi-agent handoff) / Streaming FC │
│ ├── GuardedAgent (Guardrails safety) │
│ ├── DeepResearchAgent (multi-round research + citations) │
│ ├── AgentExecutor (parallel tool calls, bounded concurrency)│
│ ├── Supervisor (LLM routes tasks to sub-agents, FINISH) │
│ ├── A2A Server/Client (Agent-to-Agent protocol) │
│ └── LangGraph (StateGraph, Subgraph, Parallel, dynamic │
│ │ interrupt/resume, history + fork time-travel)│
├─────────────────────────────────────────────────────────────┤
│ MCP Layer │
│ ├── StatelessMcpClient (HTTP POST) -> MCPToolAdapter │
│ ├── MCPServer (expose BaseTool to host) │
│ └── Full 6 primitives (resources/prompts/completion/...) │
├─────────────────────────────────────────────────────────────┤
│ Retrieval Layer │
│ ├── RAG (TextSplitter, SemanticSplitter, VectorStore) │
│ ├── BM25 (Keyword Search, AutoMerging) │
│ ├── Hybrid (BM25 + Vector, RRF/Weighted fusion, MMR) │
│ ├── Neural Rerank (Cohere/Jina) / Sentence-Window / Parent │
│ ├── HyDE / MultiQuery / Reranking / late chunking │
│ ├── CorrectiveRAG (grade + rewrite + hallucination detect) │
│ ├── AdaptiveRAG (LLM-routed retrieval strategy) │
│ ├── GraphRAG (knowledge graph + community detection) │
│ └── Loaders (Text/JSON/MD/PDF/CSV/HTML/Docx/Web/Sitemap) │
├─────────────────────────────────────────────────────────────┤
│ Storage Layer │
│ ├── Vector DB (InMemory, Qdrant, MongoDB, ChromaDB, │
│ │ Redis, SQLite, PGVector, Pinecone, File) │
│ └── Sessions (EventSessionManager + EventStore) │
├─────────────────────────────────────────────────────────────┤
│ Utility Layer │
│ ├── Memory (Buffer, Window, Summary, SummaryBuffer, Vector,│
│ │ ContextWindow[Truncate+Summarize]) │
│ ├── Chains (LLMChain, SequentialChain, RetrievalQA, ...) │
│ │ + Chain streaming (per-token output) │
│ ├── Prompts (PromptTemplate, ChatPromptTemplate, FewShot) │
│ ├── Tools (Calculator, DateTime, URLFetch, HTTP/File/SQL, │
│ │ ComputerUseTool, CodeSandbox) │
│ ├── Output Parsers │
│ ├── Token Counter (Tiktoken + Cost Tracking) │
│ ├── LLM Cache │
│ ├── Evaluation (10+ evaluators, RAGAS, golden/compare gate)│
│ ├── Tracing (Tracer + SpanGuard, InMemory/Console/OTel) │
│ └── Callbacks (LangSmith, StdOut, FileHandler, Otel) │
└─────────────────────────────────────────────────────────────┘
```
---
## Installation
```toml
[dependencies]
langchainrust = "0.24.0"
tokio = { version = "1.0", features = ["full"] }
# Optional features
langchainrust = { version = "0.24.0", features = ["mongodb-persistence"] } # MongoDB storage
langchainrust = { version = "0.24.0", features = ["qdrant-integration"] } # Qdrant vector DB
langchainrust = { version = "0.24.0", features = ["redis-storage"] } # Redis storage
langchainrust = { version = "0.24.0", features = ["sqlite-storage"] } # SQLite storage (+ SQLTool)
langchainrust = { version = "0.24.0", features = ["pgvector-storage"] } # PGVector (requires user-configured sqlx/pgvector deps)
langchainrust = { version = "0.24.0", features = ["local-embeddings"] } # Local ONNX embeddings (requires ort)
langchainrust = { version = "0.24.0", features = ["opentelemetry"] } # OpenTelemetry tracing
# PineconeStore / FileVectorStore require no feature flag, available by default
```
### Rust version requirements (MSRV)
- **Default features: Rust 1.85+** — this is the workspace MSRV, enforced in CI
and used for MSRV-aware dependency resolution (`resolver = "3"`).
- The local-model features **`local-embeddings`** (ONNX via `ort`),
**`fastembed`**, and **`local-candle`** (candle) pull crates that require a
newer toolchain (currently **Rust 1.88+**). Cargo cannot declare an MSRV per
feature, so this layering is documented rather than compile-enforced: use
these three features only with a recent stable Rust. docs.rs builds them on
the latest nightly automatically.
---
## Quick Start
```rust
use langchainrust::{OpenAIChat, OpenAIConfig, BaseChatModel};
use langchainrust::schema::Message;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let config = OpenAIConfig {
api_key: std::env::var("OPENAI_API_KEY")?,
base_url: "https://api.openai.com/v1".to_string(),
model: "gpt-3.5-turbo".to_string(),
..Default::default()
};
let llm = OpenAIChat::new(config);
let response = llm.chat(vec![
Message::system("You are a helpful assistant."),
Message::human("What is Rust?"),
], None).await?;
println!("{}", response.content);
Ok(())
}
```
### Multi-Provider Support
```rust
use langchainrust::{
DeepSeekChat, MoonshotChat, ZhipuChat, QwenChat,
AnthropicChat, OllamaChat,
};
let deepseek = DeepSeekChat::from_env();
let moonshot = MoonshotChat::with_model("moonshot-v1-128k");
let claude = AnthropicChat::from_env();
let ollama = OllamaChat::new("llama3.2");
```
### BM25 Keyword Search
```rust
use langchainrust::{BM25Retriever, Document};
let mut retriever = BM25Retriever::new();
retriever.add_documents_sync(vec![
Document::new("Rust is a systems programming language"),
Document::new("Python is a scripting language"),
]);
let results = retriever.search("systems programming", 3);
for result in results {
println!("Document: {}", result.document.content);
println!("Score: {}", result.score);
}
```
More examples in [Usage Guide](https://github.com/atliliw/langchainrust/blob/main/docs/USAGE_EN.md).
---
## Examples
The `examples/` directory provides 42 runnable examples covering core functionality:
| basic | chat / streaming / multi_provider / token_counter / quick_start / responses_api / batch_api / sandbox | Yes |
| agent | function_calling / multi_tool / assistants / handoffs / plan_execute / deep_research / extended_thinking | Yes |
| agent SSE | agent_sse_server (axum + SSE) | Yes (`AGENT_SSE_API_KEY`) |
| rag | bm25_search / document_loaders / file_vectorstore / semantic_splitter / adaptive_rag / corrective_rag / graph_rag | No |
| langgraph | basic_graph / conditional_edge | No |
| memory | buffer_memory / context_window / sessions / vectorstore_memory | No |
| chains | llm_chain / sequential_chain | Yes |
| lcel | lcel_pipe / lcel_compose | pipe: No / compose: Yes |
| evaluation | evaluation / ragas_eval | evaluation: No / ragas_eval: Yes |
| guardrails | guardrails | No |
| mcp | mcp_http_server / mcp_stdio_server / mcp_server | No |
| a2a | a2a_http_server | Yes |
| otel | otel_tracing / otlp_tracing (needs `--features otlp`) | No |
Examples requiring API keys read from environment variables:
```bash
export OPENAI_API_KEY="your-key"
cargo run --example basic_chat
```
Examples without API keys (BM25 / LangGraph / Memory / Loader) can run directly — great for quick exploration.
---
## Documentation
| [Usage Guide](https://github.com/atliliw/langchainrust/blob/main/docs/USAGE_EN.md) | Detailed usage for all components |
| [API Docs](https://docs.rs/langchainrust) | Rust API documentation |
| [Changelog](https://github.com/atliliw/langchainrust/blob/main/CHANGELOG.md) | Release history and breaking changes |
---
## Testing
```bash
cargo test
```
---
## Contributing
Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md).
---
## License
MIT or Apache-2.0, at your option.