LEANN-rs

Rust implementation of LEANN -- a lightweight vector database and RAG system that achieves 97% storage reduction through graph-based selective embedding recomputation.

Overview

LEANN-rs is a full rewrite of the Python LEANN system in Rust, providing:

• Pure Rust HNSW engine -- build and search without FAISS or any C++ dependencies
• Embedding recomputation -- prune stored embeddings and recompute on-the-fly via ZMQ, reducing index size by ~97%
• Multiple embedding backends -- OpenAI, Ollama (pipelined async), Gemini APIs (ONNX local inference planned)
• RAG pipeline -- search + LLM chat with Ollama, OpenAI, and Anthropic providers
• Python bindings -- PyO3-based native module, drop-in replacement for the Python version
• HTTP server -- Axum-based REST API for search

Crates

Quick Start

Build

cargo build --release

Build an index

# From a directory of text/code files
leann build my-index --docs ./documents/ --embedding-model text-embedding-3-small --embedding-mode openai

# With Ollama (local, no API key needed)
leann build my-index --docs ./documents/ --embedding-model nomic-embed-text --embedding-mode ollama

# From multiple directories and individual files
leann build my-index --docs ./src ./tests ./config.json

# Build with AST-aware code chunking
leann build my-code --docs ./src --use-ast-chunking

# Force rebuild with custom file types
leann build my-docs --docs ./ --file-types .txt,.pdf,.pptx --force

# Disable compact storage / recomputation
leann build my-index --docs ./data --no-compact --no-recompute

Search

leann search my-index "how does HNSW search work"
leann search my-index "query" --top-k 10 --show-metadata

RAG Q&A

# Single question
leann ask my-index "What is embedding recomputation?"

# Interactive mode
leann ask my-index --interactive

# With a specific LLM provider
leann ask my-index "question" --llm openai --model gpt-4o --api-key $OPENAI_API_KEY

# With thinking budget for reasoning models
leann ask my-index "complex question" --thinking-budget high

ReAct Agent

# Multi-turn retrieval and reasoning
leann react my-index "complex question requiring multiple searches"

Other commands

leann list                         # List all indexes in current directory
leann remove my-index              # Delete an index and all its files
leann remove my-index --force      # Delete without confirmation
leann watch my-index               # Check for file changes since last build
leann serve --port 8080            # Start HTTP server

Global options

leann -v build my-index --docs ./src  # Verbose output (including backend logs)
leann -q search my-index "query"      # Quiet mode (suppress non-essential output)

HTTP Server

Start the standalone server:

# Via the server binary
LEANN_INDEX_DIR=./indexes leann-server

# Or via the CLI
leann serve --port 8080

Endpoints

Search request body:

{
  "query": "your search query",
  "top_k": 5,
  "complexity": 64,
  "use_grep": false
}

Python Bindings

The leann-python crate provides native Python bindings via PyO3, with .pyi type stubs for IDE autocomplete/typecheck. Build with maturin:

cd crates/leann-python
maturin develop --release

Run binding tests:

cd crates/leann-python && cargo test --test test_maturin

Usage:

from leann import LeannBuilder, LeannSearcher, LeannChat

# Build an index
builder = LeannBuilder("nomic-embed-text", embedding_mode="ollama")
builder.add_text("HNSW is a graph-based ANN algorithm.", {"source": "docs"})
builder.add_text("Embedding recomputation saves storage.", {"source": "docs"})
builder.build_index("./my-index")

# Search
searcher = LeannSearcher("./my-index")
results = searcher.search("graph algorithms", top_k=5)
for r in results:
    print(f"[{r.score:.3f}] {r.text[:80]}")

# RAG Q&A
chat = LeannChat("./my-index", llm_config={"type": "ollama", "model": "llama3:8b"})
answer = chat.ask("What is HNSW?")
print(answer)

Architecture

HNSW Engine

The core HNSW implementation in leann-core/src/hnsw/ includes:

• simd.rs -- NEON (aarch64) and AVX2 (x86_64) SIMD-optimized L2 and inner product distance with batch-4 processing, plus FlatMinHeap/FlatMaxHeap and VisitedList data structures
• build.rs -- FAISS-style insert algorithm with parallel construction (rayon), monomorphized distance functions, early termination, and deterministic RNG seed support
• search.rs -- Two-phase beam search with SIMD batch-4 distance, flat heaps, generation-counter visited tracking, and support for both stored-vector and recompute modes
• csr.rs -- Compact CSR format conversion for pruned indexes
• io.rs -- Binary serialization for both standard and compact graph formats

Embedding Recomputation

Instead of storing all embedding vectors (which dominate index size), LEANN prunes them and recomputes distances on-the-fly during search via a ZMQ REQ/REP protocol:

1. The search algorithm encounters a pruned node
2. It sends node IDs + query vector to the embedding server via ZMQ
3. The server recomputes embeddings and returns distances
4. Search continues with fresh distances

This achieves ~97% storage reduction with minimal latency impact.

Index File Format

A LEANN index consists of:

Configuration

Environment Variables

Benchmarks

The pure-Rust HNSW engine matches or exceeds FAISS C++ performance: 3.3x geometric mean speedup across 25 benchmarks (distance, build, search, recompute, full pipeline, passage lookup, index size). Distance computations are 9-111x faster via SIMD; search is on par; passage lookups are 2.5-2.8x faster; index files are 85% smaller.

See RUST_PERFORMANCE.md for full results, methodology, and reproduction instructions.

Quick start:

# All-in-one comparison (Rust vs Python/FAISS)
bash benchmarks/compare_rust_python.sh

# Criterion benchmarks only (HTML reports in target/criterion/)
cargo bench --package leann-core

Development

# Run tests
cargo test

# Check all crates
cargo check

# Build release
cargo build --release

# Build Python bindings
cd crates/leann-python && maturin develop

# Test Python bindings (maturin develop + pytest)
cd crates/leann-python && cargo test --test test_maturin

License

MIT