# Cerebro User Guide
Welcome to the Cerebro User Guide! This document covers how to implement the memory protocol in your application.
## 1. Initializing the Engine
Cerebro is designed to be modular. You might mix and match your compute backends and storage layers based on your privacy and scale needs.
```rust
use std::sync::Arc;
use cerebro::prelude::*;
use cerebro::chunker::semantic::HtmlSemanticChunker;
use cerebro::compute::local::LocalEmbedder;
use cerebro::storage::qdrant::QdrantVectorStore;
#[tokio::main]
async fn main() {
// 1. Semantic Chunking (HTML aware)
let chunker = Arc::new(HtmlSemanticChunker::new(1024));
// 2. Local Privacy-First Embeddings (runs on the CPU)
// Requires feature = ["local_models"]
let embedder = Arc::new(LocalEmbedder::new().await.unwrap());
// 3. Persistent Storage (requires feature = ["qdrant"]); 384 = embedding size
let store = Arc::new(QdrantVectorStore::new("http://localhost:6334", "memories", 384).await.unwrap());
// Connect the neurons!
let engine = MemoryEngine::new(chunker, embedder, store);
println!("Brain is online.");
}
```
## 2. Ingesting Complex Data
Cerebro supports basic text, HTML, and PDF ingestion.
```rust
use cerebro::ingest::pdf::PdfIngestor;
// Ingest a PDF file directly
let pdf_ingestor = PdfIngestor::new();
let docs = pdf_ingestor.ingest("whitepaper.pdf").await.unwrap();
for doc in docs {
engine.ingest_document(doc).await.unwrap();
}
```
## 3. Hybrid Search & RRF
Cerebro automatically performs Hybrid Search (Full-Text + Vector) when using the `PgVectorStore`, merging results via **Reciprocal Rank Fusion (RRF)** for maximum accuracy.
```rust
// The query automatically triggers the hybrid merge logic
let results = engine.query("neural architecture", 5).await.unwrap();
```
## 4. Advanced Features
### Consolidation Worker (experimental)
`MemoryEngine::start_consolidation_loop` starts a background worker that scans the stored memories on a fixed interval. If you build a `ConsolidationWorker` with `with_holographic_compression(provider)` and more than 10 memories are stored, it asks the LLM to summarise up to 20 of them into one "axiom" and prints it. The axiom is not stored yet, and pruning and merging are not implemented.
### Optional Feature Flags
To keep your binary lean, most integrations are opt-in:
- `local_models`: Enable `candle`-based local inference.
- `qdrant`: Enable distributed Qdrant storage.
- `pdf`: Enable PDF extraction via `pdf-extract`.
- `graph`: Enable Neo4j Knowledge Graph persistence and Graphify abstractions.
- `spatial`: Enable the experimental `SpatialMemoryStore` (nearest-neighbour lookup over 3D coordinates).
- `python` / `wasm`: Enable the FFI binding scaffolding (placeholder classes only).
## 5. FFI & Language Support
> **Status:** both bindings are scaffolding only. The classes exist, but `ingest` and `search` are not wired to the engine yet, and there is no `maturin`/PyPI packaging.
### Python
The `python` feature compiles a placeholder `Cerebro` class:
```python
import cerebro
engine = cerebro.Cerebro()
engine.ingest("Some data...")
```
### WASM
Compile with `--features wasm` to get a placeholder `CerebroWasm` class for JS.
```javascript
import init, { CerebroWasm } from './pkg/cerebro.js';
const engine = new CerebroWasm();
```
## 6. Model Context Protocol (MCP)
Cerebro features a native `stdio`-based MCP server. This allows AI development environments (like Cursor IDE, Claude Desktop, and others) to connect to Cerebro to natively ingest memories, search the vector store, or run a swarm.
### Compiling the Server
```bash
cargo build --release --bin cerebro-mcp
```
The compiled server binary will be located at `target/release/cerebro-mcp`.
### Configuring your IDE
Add the absolute path to your MCP client configuration (e.g., `claude_desktop_config.json`):
```json
{
"mcpServers": {
"cerebro": {
"command": "/absolute/path/to/cerebro/target/release/cerebro-mcp",
"args": []
}
}
}
```
The server automatically exposes tools like `ingest_memory`, `search_memory`, and `run_swarm`.
## 7. SwarmForge — Multi-Agent Swarming
SwarmForge turns Cerebro into a multi-agent orchestration engine. Agents collaborate through Cerebro's three memory tiers:
- **Working Memory** (`KVStore`) — fast per-agent state
- **Semantic Memory** (`MemoryEngine`) — agents commit outputs as searchable documents
- **Episodic Memory** — per-agent conversation histories
### Setting Up a Swarm
```rust
use cerebro::prelude::*;
use cerebro::swarm::prelude::*;
use std::sync::Arc;
// 1. Build the Cerebro memory bus
let engine = Arc::new(MemoryEngine::new(
Arc::new(RecursiveCharacterChunker::new(512, 50)),
Arc::new(MockEmbedder::new(8)),
Arc::new(MemoryVectorStore::new()),
));
let memory = Arc::new(CerebroMemoryBus::new(engine, Arc::new(MemoryKVStore::new())));
// 2. Create orchestrator
let mut orch = SwarmOrchestrator::new(memory);
```
### Registering Agents with Any LLM
Each agent can use a different LLM provider:
```rust
// Agent using local Ollama
orch.register_agent(AgentConfig {
id: "researcher".into(),
name: "Research Agent".into(),
system_prompt: "You research topics thoroughly.".into(),
model: LlmProvider::Ollama { model: "llama3".into(), base_url: "http://localhost:11434".into() },
tools: vec![], handoff_targets: vec![], max_steps: 10,
});
// Agent using Anthropic Claude
orch.register_agent(AgentConfig {
id: "writer".into(),
name: "Writer Agent".into(),
system_prompt: "You write clear, compelling content.".into(),
model: LlmProvider::Anthropic {
model: "claude-sonnet-4-20250514".into(),
api_key: std::env::var("ANTHROPIC_API_KEY").unwrap(),
max_tokens: 4096,
},
tools: vec![], handoff_targets: vec![], max_steps: 10,
});
// Agent using Groq (via OpenAI-compatible)
orch.register_agent(AgentConfig {
id: "reviewer".into(),
name: "Review Agent".into(),
system_prompt: "You review content for quality.".into(),
model: LlmProvider::OpenAICompatible {
model: "llama-3.3-70b-versatile".into(),
api_key: std::env::var("GROQ_API_KEY").unwrap(),
base_url: "https://api.groq.com/openai/v1".into(),
provider_name: Some("groq".into()),
},
tools: vec![], handoff_targets: vec![], max_steps: 10,
});
```
### Executing Swarm Patterns
#### Sequential Pipeline
```rust
let result = orch.execute(
SwarmPattern::Sequential {
agent_order: vec!["researcher".into(), "writer".into(), "reviewer".into()],
},
"Write an article about Rust's memory safety model",
).await.unwrap();
```
#### Parallel Fan-Out / Fan-In
```rust
let result = orch.execute(
SwarmPattern::Parallel {
agents: vec!["security".into(), "perf".into(), "style".into()],
merger: "synthesizer".into(),
},
"Review this code for all issues",
).await.unwrap();
```
#### Hierarchical Supervisor
```rust
let result = orch.execute(
SwarmPattern::Hierarchical {
supervisor: "lead".into(),
workers: vec!["backend".into(), "frontend".into(), "testing".into()],
},
"Plan the implementation of user authentication",
).await.unwrap();
```
### Tool Calling & ReAct
Agents can interact with external systems using tools dynamically executed by the orchestrator:
```rust
let my_tool = ... // Implements AgentTool
orch.register_tool(my_tool);
orch.register_agent(AgentConfig {
id: "researcher".into(),
name: "Research Agent".into(),
system_prompt: "Research topics thoroughly.".into(),
model: LlmProvider::Ollama { model: "llama3".into(), base_url: "http://localhost:11434".into() },
tools: vec![ToolDefinition { name: "search".into(), description: "Search the web".into(), parameters_schema: json!({}) }],
handoff_targets: vec![], max_steps: 10,
});
```
### CLI & HTTP Gateway
If you compile with the `cli` and `api` features, you gain access to the standalone gateway binary `cerebro-swarm`:
```bash
cargo run --bin cerebro-swarm --features api,cli -- serve --port 3000
```
This boots an Axum gateway at `POST /api/swarm/execute` which accepts JSON payloads to trigger patterns remotely.
### Supported LLM Providers
| Ollama | `LlmProvider::Ollama` | Any local model (Llama 3, Mistral, Phi, Gemma, etc.) |
| OpenAI | `LlmProvider::OpenAI` | GPT-4o, GPT-4, o3, etc. |
| Gemini | `LlmProvider::Gemini` | Gemini Pro, Flash, Ultra |
| Anthropic | `LlmProvider::Anthropic` | Claude 4, Sonnet, Haiku |
| Any OpenAI-compatible | `LlmProvider::OpenAICompatible` | Groq, Together, Mistral, DeepSeek, Fireworks, LM Studio, vLLM, Anyscale, etc. |
---