# Project Backlog & Future Enhancements
This document tracks upcoming tasks, feature requests, and major architectural enhancements for **Cerebro**. Items are prioritized from top to bottom.
## Phase 1: The Foundation
- [x] **Core Trait Architecture (`cerebro::traits`)**: Design and implement the universal trait system — `Chunker`, `Embedder`, `VectorStore`, `KVStore`, `Ingestor` — that all backends plug into.
- [x] **Core Data Models (`cerebro::models`)**: Define the `Document`, `Chunk`, `Node`, and `Metadata` structures with full Serde serialization support and UUID-based identity.
- [x] **Error Hierarchy (`cerebro::traits`)**: Build a unified `CerebroError` enum via `thiserror` covering Storage, Embedding, and Ingestion error domains.
- [x] **In-Memory Vector Store (`cerebro::storage`)**: Implement `MemoryVectorStore` — a fast, lock-guarded HashMap with naive cosine similarity for zero-dependency local testing.
- [x] **Mock Embedder (`cerebro::compute`)**: Build a deterministic mock embedder for offline testing without API keys.
- [x] **Working Memory KVStore (`cerebro::storage`)**: Implement the working memory fast state retrieval layer (e.g., `MemoryKVStore`).
## Phase 2: The Core Engine
- [x] **Recursive Character Chunker (`cerebro::chunker`)**: Implement a character-boundary-safe recursive text splitter with configurable chunk size and overlap, including Unicode safety.
- [x] **Memory Engine (`cerebro::engine`)**: Build the unified `MemoryEngine` struct that ties together `Chunker`, `Embedder`, and `VectorStore` into a single `ingest_document()` → `query()` pipeline.
- [x] **Comprehensive Test Suite**: Comprehensive tests across all modules covering models, serialization, chunker edge cases, cosine similarity, storage CRUD, and full engine integration.
- [x] **Project Documentation**: Author README, ARCHITECTURE.md, USER_GUIDE.md, and CHANGELOG with Mermaid data pipeline diagrams.
- [x] **crates.io Publication Readiness**: Add MIT License, full Cargo metadata (description, repository, keywords, categories).
## Phase 3: The Enterprise Backends
- [x] **Remote Compute Integration (`cerebro::compute`)**: Wrapper layers calling OpenAI APIs for embedding generation to quickly scale vector models.
- [x] **PgVector Integration (`cerebro::storage`)**: Build the `Sqlx` powered PostgreSQL vector store.
- [x] **Qdrant Integration (`cerebro::storage`)**: Build the high-volume distributed vector store driver.
- [x] **Claude APIs Integration (`cerebro::compute`)**: Add support for Anthropic's Claude to generate embeddings or handle inference.
## Phase 4: The Interoperability Matrix
- [x] **Native MCP Server (`cerebro-mcp`)**: Write the stdio/SSE server allowing desktop applications (like Cursor and Claude Desktop) to connect directly to Cerebro without code modifications. (Currently basic, needs configuration upgrades).
- [x] **Python Bindings**: Implement `PyO3` layer and configure `maturin` builds so agents can install Cerebro via `pip install cerebro`.
- [x] **WASM & Edge Bindings**: Implement `wasm-bindgen` enabling Cerebro to run in Cloudflare edge workers and browsers natively.
## Phase 5: Advanced AI Cognitive Features
- [x] **Active Memory Consolidation Engine**: Introduce background Tokio worker loops that act as the AI's "sleep cycle", constantly pruning dead memory, merging similar chunks, and refining the vector indices without blocking the main event loops.
- [x] **PDF & Complex Ingestion Integration**: Sync up with structural parsing libraries (like Docling) or native Rust `pdfium-render` so the engine can ingest complex tables, images, and non-text layouts.
- [x] **Semantic / HTML Bound Chunking**: Upgrade the chunker from recursive sizes to chunking based on semantic sentence boundaries and HTML layout tags.
- [x] **Local Fast Embedders**: Incorporate `candle-core` (HuggingFace's Rust ML framework) to allow generating embeddings directly on user CPUs natively without API calls, prioritizing 100% data privacy.
## Phase 6: Enterprise Cognitive Upgrades
- [x] **Hybrid Search Engine**: Combine Sparse (BM25 or SPLADE keyword search) with Dense (Vector similarity) retrieving. The engine will rank results using Reciprocal Rank Fusion (RRF) for significantly higher accuracy.
- [x] **Knowledge Graph Entity Extraction**: When chunks are ingested, run them through an LLM to extract node entities and relationships, storing them alongside the vectors in a format compatible with Cypher/Neo4j graph databases.
- [x] **Multi-Modal Memory Construction**: Abstract the `Document` formatting to accept binary image slices and use multimodal LLMs (like CLIP models) to generate embedding vectors directly from visual data, unified with the text chunks.
---