Crate brainwires_storage 

Brainwires Storage — backend-agnostic persistent storage for the Brainwires Agent Framework.

This crate provides conversation storage with semantic search, document ingestion with hybrid retrieval, three-tier memory hierarchy, image analysis storage, cross-process lock coordination, and reusable plan templates.

Unified Database Layer (databases module)

One struct per database, one shared connection, implementing one or both of the core traits:

• StorageBackend — generic CRUD + vector search for domain stores
• VectorDatabase — RAG embedding storage with hybrid search

Database backends

Backend	Struct	StorageBackend	VectorDatabase	Feature
LanceDB	LanceDatabase	YES	YES	lance-backend (default)
PostgreSQL	PostgresDatabase	YES	YES	postgres-backend
MySQL	MySqlDatabase	YES	NO	mysql-backend
SurrealDB	SurrealDatabase	YES	YES	surrealdb-backend
Qdrant	QdrantDatabase	NO	YES	qdrant-backend
Pinecone	PineconeDatabase	NO	YES	pinecone-backend
Milvus	MilvusDatabase	NO	YES	milvus-backend
Weaviate	WeaviateDatabase	NO	YES	weaviate-backend
NornicDB	NornicDatabase	NO	YES	nornicdb-backend

Backends that implement both traits share a single connection — construct once, wrap in Arc, and pass to both domain stores and RAG subsystem.

Core Infrastructure

• FastEmbedManager — text embeddings via FastEmbed ONNX model (all-MiniLM-L6-v2, 384 dimensions)
• CachedEmbeddingProvider — LRU-cached embedding provider (1000 entries)

Domain Stores (stores module)

• MessageStore — conversation messages with vector search and TTL expiry
• ConversationStore — conversation metadata with create-or-update semantics
• TaskStore / AgentStateStore — task and agent state persistence
• PlanStore — execution plan storage with markdown export
• TemplateStore — reusable plan templates with {{variable}} substitution
• LockStore — SQLite-backed cross-process lock coordination

Document Management

• DocumentStore — hybrid search (vector + BM25 via RRF)
• DocumentProcessor — PDF, DOCX, Markdown, plain text ingestion
• DocumentChunker — paragraph/sentence-aware segmentation
• DocumentMetadataStore — hash-based deduplication

Image Storage

• ImageStore — analyzed images with semantic search over descriptions

Tiered Memory

• TieredMemory — three-tier memory hierarchy (hot/warm/cold)
• SummaryStore — compressed message summaries (warm tier)
• FactStore — key facts extraction (cold tier)
• TierMetadataStore — access tracking and importance scoring

Feature Flags

Feature	Default	Description
native	Yes	LanceDB backend + FastEmbed + SQLite locks + all native stores
lance-backend	Yes (via native)	LanceDB embedded vector database
postgres-backend	No	PostgreSQL + pgvector
mysql-backend	No	MySQL / MariaDB
surrealdb-backend	No	SurrealDB with native MTREE vector search
qdrant-backend	No	Qdrant vector search
pinecone-backend	No	Pinecone cloud vectors
milvus-backend	No	Milvus vectors
weaviate-backend	No	Weaviate search engine
nornicdb-backend	No	NornicDB graph + vector
wasm	No	WASM-compatible (pure types only)

Re-exports

pub use databases::BackendCapabilities;

pub use databases::traits::StorageBackend;

pub use databases::types::record_get;

pub use databases::types::FieldDef;

pub use databases::types::FieldType;

pub use databases::types::FieldValue;

pub use databases::types::Filter;

pub use databases::types::Record;

pub use databases::types::ScoredRecord;

pub use databases::LanceDatabase;

pub use image_types::ImageFormat;

pub use image_types::ImageMetadata;

pub use image_types::ImageSearchRequest;

pub use image_types::ImageSearchResult;

pub use image_types::ImageStorage;

pub use stores::template_store::PlanTemplate;

pub use stores::template_store::TemplateStore;

pub use embeddings::CachedEmbeddingProvider;

pub use embeddings::EmbeddingProvider;

pub use embeddings::FastEmbedManager;

pub use file_context::FileChunk;

pub use file_context::FileContent;

pub use file_context::FileContextManager;

pub use stores::conversation_store::ConversationMetadata;

pub use stores::conversation_store::ConversationStore;

pub use stores::fact_store::FactStore;

pub use stores::image_store::ImageStore;

pub use stores::lock_store::LockRecord;

pub use stores::lock_store::LockStats;

pub use stores::lock_store::LockStore;

pub use stores::message_store::MessageMetadata;

pub use stores::message_store::MessageStore;

pub use stores::plan_store::PlanStore;

pub use stores::summary_store::SummaryStore;

pub use stores::task_store::AgentStateMetadata;

pub use stores::task_store::AgentStateStore;

pub use stores::task_store::TaskMetadata;

pub use stores::task_store::TaskStore;

pub use stores::tier_metadata_store::TierMetadataStore;

pub use tiered_memory::CanonicalWriteToken;

pub use tiered_memory::MemoryAuthority;

pub use tiered_memory::MemoryTier;

pub use tiered_memory::MultiFactorScore;

pub use tiered_memory::TieredMemory;

pub use tiered_memory::TieredMemoryConfig;

pub use tiered_memory::TieredSearchResult;

pub use brainwires_core;

Modules

bm25_search

BM25 keyword search using Tantivy.

databases

Unified database layer — one struct per database, shared connection, implementing StorageBackend and/or VectorDatabase. Unified database layer for the Brainwires storage system.

embeddings

Embedding provider for vector operations. Embedding Provider

file_context

File Context Manager

glob_utils

Glob pattern matching utilities. Glob pattern matching utilities for path filtering

image_types

Image Analysis Types

paths

Platform-specific path utilities.

prelude

Prelude module for convenient imports

stores

Domain stores for conversation, message, task, plan, and other data. Domain stores for conversation, message, task, plan, and other data persistence.

tiered_memory

Tiered Memory Storage System

Traits

EmbeddingProviderTrait

Trait for text embedding generation.