Crate brainwires_storage 

Brainwires Storage — backend-agnostic persistent storage primitives for the Brainwires Agent Framework.

This crate provides the generic abstractions: a StorageBackend trait, the per-backend connections (LanceDatabase, PostgresDatabase, …), embeddings (CachedEmbeddingProvider), BM25 keyword search, file chunking, and image metadata types. Domain-shaped stores (ConversationStore, MessageStore, PlanStore, LockStore, …) and the tiered hot/warm/cold memory orchestration moved out:

• brainwires-memory — MessageStore, SummaryStore, FactStore, MentalModelStore, TierMetadataStore, TieredMemory.
• brainwires-cli crate::storage — ConversationStore, TaskStore / AgentStateStore, PlanStore, TemplateStore, LockStore, ImageStore, PersistentTaskManager.

Unified Database Layer (databases module)

One struct per database, one shared connection, implementing one or both of the core traits:

• StorageBackend — generic CRUD + vector search for domain stores
• VectorDatabase — RAG embedding storage with hybrid search

Database backends

Backend	Struct	StorageBackend	VectorDatabase	Feature
LanceDB	LanceDatabase	YES	YES	lance-backend (default)
PostgreSQL	PostgresDatabase	YES	YES	postgres-backend
MySQL	MySqlDatabase	YES	NO	mysql-backend
SurrealDB	SurrealDatabase	YES	YES	surrealdb-backend
Qdrant	QdrantDatabase	NO	YES	qdrant-backend
Pinecone	PineconeDatabase	NO	YES	pinecone-backend
Milvus	MilvusDatabase	NO	YES	milvus-backend
Weaviate	WeaviateDatabase	NO	YES	weaviate-backend
NornicDB	NornicDatabase	NO	YES	nornicdb-backend

Backends that implement both traits share a single connection — construct once, wrap in Arc, and pass to both domain stores and RAG subsystem.

Core Infrastructure

• FastEmbedManager — text embeddings via FastEmbed ONNX model (all-MiniLM-L6-v2, 384 dimensions)
• CachedEmbeddingProvider — LRU-cached embedding provider (1000 entries)
• BM25Search — keyword search via Tantivy
• FileContextManager — file chunking / context primitives

Image Types

• ImageFormat, ImageMetadata, ImageSearchRequest, ImageSearchResult, ImageStorage — pure types reused by the ImageStore that lives in brainwires-cli::storage.

Feature Flags

Feature	Default	Description
native	Yes	LanceDB backend + FastEmbed + file context + native primitives
lance-backend	Yes (via native)	LanceDB embedded vector database
postgres-backend	No	PostgreSQL + pgvector
mysql-backend	No	MySQL / MariaDB
surrealdb-backend	No	SurrealDB with native MTREE vector search
qdrant-backend	No	Qdrant vector search
pinecone-backend	No	Pinecone cloud vectors
milvus-backend	No	Milvus vectors
weaviate-backend	No	Weaviate search engine
nornicdb-backend	No	NornicDB graph + vector
wasm	No	WASM-compatible (pure types only)

Re-exports

pub use error::StorageError;

pub use databases::BackendCapabilities;

pub use databases::traits::StorageBackend;

pub use databases::types::record_get;

pub use databases::types::FieldDef;

pub use databases::types::FieldType;

pub use databases::types::FieldValue;

pub use databases::types::Filter;

pub use databases::types::Record;

pub use databases::types::ScoredRecord;

pub use databases::LanceDatabase;

pub use image_types::ImageFormat;

pub use image_types::ImageMetadata;

pub use image_types::ImageSearchRequest;

pub use image_types::ImageSearchResult;

pub use image_types::ImageStorage;

pub use embeddings::CachedEmbeddingProvider;

pub use embeddings::FastEmbedManager;

pub use brainwires_core;

Modules

bm25_search

BM25 keyword search using Tantivy. Used by Lance backend for hybrid vector + keyword search; consumed by brainwires-rag and other indexers.

databases

Unified database layer — one struct per database, shared connection, implementing StorageBackend and/or VectorDatabase. Unified database layer for the Brainwires storage system.

embeddings

Embedding provider for vector operations. Embedding Provider

error

Structured error taxonomy. See StorageError.

glob_utils

Glob pattern matching utilities. Used by every database backend’s path/include filtering. Glob pattern matching utilities for path filtering

image_types

Image-storage type definitions (concrete ImageStore lives in brainwires-cli). Image Analysis Types

prelude

Prelude module for convenient imports of the primitive surface.

Traits

EmbeddingProvider

Trait for text embedding generation.