# Minni
Local memory and codebase indexing tool for AI agents. Built with Rust for speed and portability.
## Features
### 🔍 **Hybrid Search: BM25 + Neural Re-ranking**
Minni uses layered retrieval for local speed and precision:
1. **Stage 1: BM25 (Fast lexical retrieval)**
- Code-aware tokenization (handles camelCase, snake_case)
- Keyword search powered by Tantivy
- Produces strong lexical candidates quickly
2. **Stage 2: Dense semantic candidates (Optional)**
- Uses a MiniLM dense model
- ANN candidate index for local sublinear retrieval
3. **Stage 3: Neural re-ranking (Optional precision boost)**
- Semantic re-ranking with a MiniLM cross-encoder
- Runs on CPU via bundled ONNX runtime, no GPU required
- Re-ranks only top candidates for efficiency
- Downloads automatically on first run
**Result**: fast default search, with semantic improvements when models are available.
### 💾 **Session Context Persistence**
- Save/load session contexts by name or ID
- Contexts stored per-project with SQLite
- Store notes, file references, tasks, decisions, findings
- Resume work exactly where you left off
### 📚 **Smart Codebase Indexing**
- Tree-sitter based semantic code parsing
- Incremental indexing (only re-indexes changed files)
- Supports: **Rust, Python, JavaScript, TypeScript, Go, C, C++, Java, C#**
- Extracts functions, classes, methods as searchable chunks
## Installation
Minni is published on [crates.io](https://crates.io/crates/minni) and supports **Windows, macOS (ARM), and Linux**.
```bash
cargo install minni
```
Requires [Rust](https://rustup.rs). The binary is installed to `~/.cargo/bin/` and available globally. Binary size is ~37MB (ONNX runtime bundled). Models download automatically on first use into `~/.minni/models/`.
> **Windows (MSVC):** If you hit linker error LNK2038 (CRT mismatch), set the runtime flags before installing:
> ```powershell
> $env:CFLAGS="/MD"; $env:CXXFLAGS="/MD"; cargo install minni
> ```
> This affects some VS BuildTools installations where `esaxx-rs` defaults to `/MT` while `ort-sys` uses `/MD`.
### Build from source
```bash
git clone <repo-url>
cd minni
cargo install --path .
```
## AI Agent Skill
Minni ships a reusable skill (`SKILL.md`) following the [open agent skills standard](https://agentskills.io). The skill is committed to this repository at `.agents/skills/minni/SKILL.md` and is auto-discovered by most agent clients at that path.
For user-level installation (available in every project), copy or symlink to the client's skills directory:
| Codex (OpenAI) | `~/.agents/skills/minni/` |
| Gemini CLI | `~/.gemini/skills/minni/` |
| GitHub Copilot | `~/.copilot/skills/minni/` |
| Claude Code | `~/.claude/skills/minni/` |
| OpenCode | `~/.config/opencode/skills/minni/` |
Copy `.agents/skills/minni/` to the appropriate directory for your client.
## Quick Start
```bash
# Initialize in your project
cd your-project
minni init
# Index the codebase
minni index
# Search (uses BM25 + optional dense/re-ranking)
minni search "authentication handler"
minni search "database connection"
# Manage context
minni context save my-session "Working on auth"
minni context list
minni context load my-session
# Check status
minni status
```
## How It Works
### Search Architecture
```
Query: "authentication handler"
↓
┌──────────────────────────────────────────┐
│ BM25 Index (Tantivy) │
│ • Tokenize: [auth, handler] │
│ • Search full-text index │
│ • Return top lexical candidates │
└──────────────────────────────────────────┘
↓
┌──────────────────────────────────────────┐
│ Dense ANN Retrieval (Optional) │
│ • Embed query with MiniLM │
│ • Probe ANN buckets for top semantic ids │
└──────────────────────────────────────────┘
↓
Combined candidate set
↓
┌──────────────────────────────────────────┐
│ Neural Re-ranker (Optional) │
│ • Load MiniLM model (first use) │
│ • Score query-doc pairs semantically │
│ • Re-sort by relevance │
└──────────────────────────────────────────┘
↓
Final top 10 results
```
### Why Hybrid Search?
| **Pure Embeddings** | Slower | ✅ Excellent | Medium |
| **Pure BM25** | Fastest | ❌ Keyword only | None |
| **Hybrid (Minni)** | Fast | ✅ Strong | Medium |
**Hybrid search gives you**:
- BM25 speed for initial retrieval
- Neural precision for final ranking
- Best of both worlds
### Code-Aware Tokenization
Minni understands code structure:
```rust
// Input: getUserName
// Tokens: [get, user, name]
// Input: handle_http_request
// Tokens: [handle, http, request]
// Query: "get user" matches getUserName ✅
```
## Data Storage
All data stored in `.minni/` directory:
```
.minni/
├── minni.db # SQLite: chunks, embeddings, contexts
├── bm25_index/ # Tantivy full-text index
├── ann_index.json # Dense ANN candidate index
└── models/ # Downloaded models (optional)
├── ms-marco-MiniLM-L6-v2/ # Re-ranker model
└── all-MiniLM-L6-v2/ # Dense model
```
## CLI Commands
### `minni init`
Initialize minni in current directory. Creates `.minni/` folder.
### `minni index [--force]`
Index the codebase. Builds BM25, dense embeddings, and ANN candidates.
- `--force`: Re-index all files (ignore cache)
### `minni search <query> [--limit N]`
Search for code snippets.
- Uses BM25 + optional dense ANN candidates + optional re-ranking
- Falls back gracefully when models are unavailable
- Default limit: 10 results
### `minni context save <name> [description]`
Save current session context.
### `minni context load <id|name>`
Load a saved context.
### `minni context list`
List all saved contexts for current project.
### `minni context delete <id|name>`
Delete a saved context.
### `minni context snapshot [--name <name>]`
Snapshot current state (files, conversation, tasks).
### `minni context export <id> [--output <file>]`
Export context for sharing.
### `minni context import <file> [--name <name>]`
Import context from another session.
### `minni context show <id>`
Show detailed context information.
### `minni context add <key> <value>`
Add information to the current context.
### `minni journal`
Manage project journal.
- `show`: Show recent entries
- `note <msg>`: Add a note
- `resume`: Show context for resuming a session
- `hooks-install`: Install git hooks for auto-journaling
### `minni status`
Show indexing status, chunk count, context count.
### `minni task`
Manage implementation tasks within a context.
- `add <context> --title <title> [--description ...] [--priority ...]`
- `list <context> [--json]`
- `show <context> <seq> [--json]`
- `update <context> <seq> [--title ...] [--description ...] [--status ...] [--priority ...]`
- `todo <context> <seq> [text] [--done <todo-seq>]`
- `export <context> <seq>`
## Advanced Usage
### Model Download
On first run of `minni search`, minni can download small MiniLM models from Hugging Face into `~/.minni/models/`:
- reranker: `cross-encoder/ms-marco-MiniLM-L6-v2`
- dense: `sentence-transformers/all-MiniLM-L6-v2`
```bash
# First run downloads models if needed
minni search "query"
# → Downloading reranker model (first run only)...
# → Model: cross-encoder/ms-marco-MiniLM-L6-v2
# → Downloading model.onnx...
# → Downloading tokenizer.json...
# → Model downloaded successfully!
```
### BM25-Only Search (No Model)
If model downloads are unavailable, minni automatically falls back to BM25-only search:
```bash
# First run without model download
minni search "auth handler"
# → Reranker model unavailable — using BM25-only search.
```
BM25-only is still very fast and effective for keyword-based searches.
### Search Quality Tips
1. **Use specific terms**: `"PostgreSQL connection pool"` > `"database"`
2. **Include function names**: `"handleRequest"` finds exact matches
3. **Combine keywords**: `"auth JWT validate"` narrows results
4. **Semantic works too**: `"user authentication"` finds `verifyCredentials`
## Performance
Performance depends on repository size, hardware, and model availability.
| Initial indexing | Variable | Includes parsing + optional embedding generation |
| Incremental re-index | Usually faster | Only changed files |
| BM25 search | Very fast | Pure lexical retrieval |
| Hybrid search | Fast-to-moderate | Depends on dense/reranker availability |
## Architecture
```
minni/
├── src/
│ ├── cli/ # Command implementations
│ ├── db/ # SQLite storage
│ ├── search/ # Hybrid search engine
│ │ ├── bm25.rs # Tantivy index
│ │ ├── ann.rs # Local ANN candidate index
│ │ ├── dense.rs # Dense embeddings + similarity
│ │ ├── reranker.rs # Neural re-ranker
│ │ └── tokenizer.rs # Code-aware tokenization
│ ├── indexer/ # Tree-sitter based indexing
│ ├── models/ # Model download helpers
│ ├── task/ # Task management
│ ├── context/ # Session management
│ └── journal/ # Session journal
```
## Dependencies
- **Tantivy**: Full-text search engine (BM25)
- **tree-sitter**: Code parsing
- **ort**: ONNX runtime (for re-ranker)
- **rusqlite**: SQLite storage
## Contributing
Contributions welcome! Areas for improvement:
- [ ] More language support (Ruby, PHP, etc.)
- [ ] Better ANN tuning and index format/versioning
- [ ] Configurable retrieval/reranking weights
- [ ] Richer task and journal workflows
- [ ] Improved import/export context ergonomics
## License
MIT
## Credits
Built with:
- [Tantivy](https://github.com/quickwit-oss/tantivy) - Fast full-text search
- [tree-sitter](https://tree-sitter.github.io/) - Code parsing
- [ONNX Runtime](https://onnxruntime.ai/) - Model inference
Inspired by:
- [Shebe](https://gitlab.com/rhobimd-oss/shebe) - BM25 code search
- RAG architectures - Retrieval-augmented generation