# Halldyll Memory Model
[](https://crates.io/crates/halldyll_memory_model)
[](https://docs.rs/halldyll_memory_model)
[](https://opensource.org/licenses/MIT)
> Multi-user, multi-model distributed memory system for AI agents.
> Designed for cloud deployment with PostgreSQL + Redis backend.
## Overview
A sophisticated memory management system enabling AI agents (like Halldyll) to:
- **Store and retrieve** conversations, facts, images, and transcriptions
- **Support multiple users** with complete data isolation and RBAC
- **Manage multiple AI models** with model-specific memory scoping
- **Search semantically** using vector embeddings with pgvector
- **Cache intelligently** with Redis for performance
- **Scale horizontally** in Kubernetes/RunPods environments
## Architecture
```
┌─────────────────────────────────────────────────────┐
│ MemoryEngine (Orchestrator) │
├─────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────┐ │
│ │ User Mgmt │ │ Embedding │ │ Context │ │
│ │ (RBAC) │ │ Generation │ │ Builder │ │
│ └──────────────┘ └──────────────┘ └──────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────┐ │
│ │ Processor │ │ Searcher │ │ Models │ │
│ │ (Ingestion) │ │ (Retrieval) │ │ Registry │ │
│ └──────────────┘ └──────────────┘ └──────────┘ │
│ │
│ ┌─────────────────────────────────────────────┐ │
│ │ StoragePool (Connection Management) │ │
│ │ ┌──────────────────┐ ┌──────────────────┐ │ │
│ │ │ PostgreSQL │ │ Redis │ │ │
│ │ │ (Data Layer) │ │ (Cache Layer) │ │ │
│ │ └──────────────────┘ └──────────────────┘ │ │
│ └─────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
```
## Features
### Multi-User Support
- User account management with roles (Admin, User, Service, Viewer)
- Role-based access control (RBAC)
- Complete data isolation between users
- Audit logging for compliance
### Memory Types
- **Conversations**: Full chat history with messages
- **Facts**: Learned information with confidence scoring
- **Images**: Generated image metadata and parameters
- **Transcriptions**: Audio transcription with language detection
- **Summaries**: Session and context summaries
### Vector Search
- pgvector integration for semantic search
- Embedding generation with ONNX Runtime
- Configurable similarity thresholds
- Full-text search fallback
### Distributed Architecture
- PostgreSQL for persistent storage
- Redis for caching and session management
- Connection pooling for efficiency
- Health checks and automatic failover
### Cloud-Ready
- Docker Compose for local development
- Kubernetes-ready configuration
- RunPods compatible
- Horizontally scalable
## Installation
### Prerequisites
- Rust 1.70+
- Docker & Docker Compose (for local development)
- PostgreSQL 14+ with pgvector extension
- Redis 6+
### Quick Start
1. **Clone the repository**
```bash
git clone https://github.com/Mr-soloDev/Halldyll-Memory-Model
cd Halldyll-Memory-Model
```
2. **Set up environment**
```bash
cp .env.example .env
docker-compose up -d
```
3. **Run migrations**
```bash
sqlx migrate run --database-url postgresql://halldyll:halldyll_dev_password@localhost/halldyll_memory
```
4. **Add to your Cargo.toml**
```toml
[dependencies]
halldyll_memory_model = "0.2"
# Or from Git:
halldyll_memory_model = { git = "https://github.com/Mr-soloDev/Halldyll-Memory-Model" }
```
## Usage
```rust
use halldyll_memory_model::{Config, MemoryEngine};
use halldyll_memory_model::models::{Conversation, Message, Role};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Create and initialize engine
let config = Config::default();
let engine = MemoryEngine::new(config).await?;
engine.initialize().await?;
// Create a conversation
let mut conversation = Conversation::new("My First Conversation");
conversation.add_message(Message::new(Role::User, "Hello!"));
conversation.add_message(Message::new(Role::Assistant, "Hi there!"));
// Store memory
let user_id = "user-123";
engine.store_memory(user_id, Memory::Conversation(conversation)).await?;
// Search memories
let results = engine.search(user_id, "hello", 10).await?;
println!("Found {} memories", results.len());
Ok(())
}
```
## Configuration
### Environment Variables
```bash
# PostgreSQL
DATABASE_URL=postgresql://user:password@host:5432/db
# Redis
REDIS_URL=redis://host:6379
# Embeddings
EMBEDDING_MODEL_PATH=./models/all-MiniLM-L6-v2.onnx
EMBEDDING_DIMENSION=384
# Search
SEARCH_TOP_K=10
SIMILARITY_THRESHOLD=0.7
# Cache
CACHE_SIZE=1000
# TTL
EPHEMERAL_FACT_TTL_DAYS=30
MAX_CONTEXT_TOKENS=4000
# Database
DB_POOL_SIZE=5
# Logging
RUST_LOG=info,halldyll_memory_model=debug
```
## Module Structure
```
src/
├── core/ # Fundamental types and errors
│ ├── config.rs # Configuration
│ ├── error.rs # Error types
│ └── id.rs # ID generation
├── models/ # Data models
│ ├── conversation.rs
│ ├── fact.rs
│ ├── image.rs
│ ├── memory.rs
│ ├── transcription.rs
│ └── user_model.rs
├── user/ # User management
│ ├── manager.rs
│ └── permissions.rs
├── storage/ # Persistence layer
│ ├── pool.rs # Connection pooling
│ ├── postgres.rs # PostgreSQL operations
│ └── redis.rs # Redis caching
├── embedding/ # Vector embeddings
├── ingest/ # Data ingestion
├── retrieval/ # Memory search
├── context/ # Prompt context building
├── engine/ # Main orchestrator
└── utils/ # Utility functions
```
## Development
### Run Tests
```bash
# Unit tests
cargo test --lib
# Integration tests (requires database)
cargo test --test '*' -- --test-threads=1
# All tests with logging
RUST_LOG=debug cargo test -- --nocapture
```
### Build
```bash
# Debug
cargo build
# Release (optimized)
cargo build --release
```
### Code Quality
```bash
# Format
cargo fmt
# Lint
cargo clippy -- -D warnings
# Check
cargo check
# Full validation
cargo check && cargo fmt && cargo clippy && cargo test
```
### Generate Documentation
```bash
cargo doc --no-deps --open
```
## Deployment
### Docker
```bash
docker build -t halldyll-memory:latest .
docker run -e DATABASE_URL=postgresql://... \
-e REDIS_URL=redis://... \
halldyll-memory:latest
```
### Kubernetes
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: halldyll-memory
spec:
replicas: 2
selector:
matchLabels:
app: halldyll-memory
template:
metadata:
labels:
app: halldyll-memory
spec:
containers:
- name: memory
image: halldyll-memory:latest
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: db-credentials
key: url
- name: REDIS_URL
valueFrom:
secretKeyRef:
name: redis-credentials
key: url
```
### RunPods
Compatible with RunPods serverless and pod deployments. Ensure PostgreSQL and Redis are accessible.
## API Overview
### Memory Storage
```rust
pub async fn store_memory(
&self,
user_id: &str,
memory: Memory,
) -> MemoryResult<()>
```
### Search
```rust
pub async fn search(
&self,
user_id: &str,
query: &str,
limit: usize,
) -> MemoryResult<Vec<Memory>>
```
### User Management
```rust
pub async fn create_user(
&self,
username: String,
email: String,
role: UserRole,
) -> MemoryResult<UserModel>
```
## Performance
- Vector search: < 100ms for top-10 on 1M vectors (with proper indexing)
- Fact storage: Non-blocking, fully asynchronous
- Embedding caching: Configurable LRU cache
- Database: Connection pooling with configurable pool size
## Security
- **User Isolation**: Complete data separation per user
- **RBAC**: Role-based access control with permission checking
- **Audit Logging**: All operations logged for compliance
- **No SQL Injection**: Uses parameterized queries (sqlx)
- **Validation**: Input validation for all user data
## Testing Models
Supported model specialties:
- **Qwen2.5-14B-Instruct**: LLM for text generation
- **Qwen2.5-VL-7B-Instruct**: Vision-language for image understanding
- **RealVisXL_V5.0**: Ultra-realistic image generation
- **ControlNet/InstantID**: Image generation control and identity preservation
- **InsightFace**: Facial recognition
## Roadmap
- [ ] GraphQL API layer
- [ ] WebSocket support for real-time memory updates
- [ ] Advanced fact extraction with LLM
- [ ] Multi-language embedding support
- [ ] Hierarchical memory compression
- [ ] Import/Export functionality
- [ ] Web dashboard for memory visualization
- [ ] Model-specific memory scoping
- [ ] Distributed tracing with OTEL
## Contributing
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit changes (`git commit -m 'Add amazing feature'`)
4. Push to branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
## License
MIT License - see LICENSE file for details
## Author
**Roy Géryan** - Halldyll Creator
## Support
For issues, questions, or suggestions:
- GitHub Issues: https://github.com/Mr-soloDev/Halldyll-Memory-Model/issues
- Discussions: https://github.com/Mr-soloDev/Halldyll-Memory-Model/discussions
- Email: geryan.roy@icloud.com
---
Built with ❤️ for Halldyll and the AI community