cerebro 0.1.6

Blazing-fast, storage-agnostic semantic memory engine for AI Agents — written in pure Rust
# Cerebro User Guide

Welcome to the Cerebro User Guide! This document covers how to implement the memory protocol in your application.

## 1. Initializing the Engine

Cerebro is designed to be highly modular. You can mix and match your compute backends and storage layers based on your privacy and scale needs.

```rust
use std::sync::Arc;
use cerebro::prelude::*;
use cerebro::compute::local::LocalEmbedder;
use cerebro::storage::pgvector::PgVectorStore;

#[tokio::main]
async fn main() {
    // 1. Semantic Chunking (HTML aware)
    let chunker = Arc::new(HtmlSemanticChunker::new(1024));
    
    // 2. Local Privacy-First Embeddings (Runs 100% on CPU/Metal)
    // Requires feature = ["local_models"]
    let embedder = Arc::new(LocalEmbedder::new().await.unwrap());
    
    // 3. Scalable Production Storage
    let store = Arc::new(PgVectorStore::new("postgres://localhost/cerebro", "memories", 384).await.unwrap());

    // Connect the neurons!
    let engine = MemoryEngine::new(chunker, embedder, store);
    
    println!("Brain is online.");
}
```

## 2. Ingesting Complex Data

Cerebro supports basic text, HTML, and PDF ingestion.

```rust
use cerebro::ingest::pdf::PdfIngestor;

// Ingest a PDF file directly
let pdf_ingestor = PdfIngestor::new();
let docs = pdf_ingestor.ingest("whitepaper.pdf").await.unwrap();

for doc in docs {
    engine.ingest_document(doc).await.unwrap();
}
```

## 3. Hybrid Search & RRF

Cerebro automatically performs Hybrid Search (Full-Text + Vector) when using the `PgVectorStore`, merging results via **Reciprocal Rank Fusion (RRF)** for maximum accuracy.

```rust
// The query automatically triggers the hybrid merge logic
let results = engine.query("neural architecture", 5).await.unwrap();
```

## 4. Advanced Features

### Consolidation (The "Sleep Cycle")
Cerebro runs a background consolidation worker that autonomously prunes dead memory and refines the vector index during idle periods.

### Optional Feature Flags
To keep your binary lean, most integrations are opt-in:
- `local_models`: Enable `candle`-based local inference.
- `qdrant`: Enable distributed Qdrant storage.
- `pdf`: Enable PDF extraction via `pdf-extract`.
- `graph`: Enable Neo4j Knowledge Graph persistence.
- `python` / `wasm`: Enable FFI bindings.

## 5. FFI & Language Support

### Python
If compiled with the `python` feature, you can use Cerebro directly in your Python apps:
```python
import cerebro
engine = cerebro.Cerebro()
engine.ingest("Some data...")
```

### WASM
Targeting the browser or Edge workers? Compile with `--features wasm` to get native JS bindings.
```javascript
import init, { CerebroWasm } from './pkg/cerebro.js';
const engine = new CerebroWasm();
```

---
*Author: Suraj Kumar Nanda* | [Surajkumarnanda.com]https://Surajkumarnanda.com