Halldyll Memory Model

Multi-user, multi-model distributed memory system for AI agents. Designed for cloud deployment with PostgreSQL + Redis backend.

Overview

A sophisticated memory management system enabling AI agents (like Halldyll) to:

Store and retrieve conversations, facts, images, and transcriptions
Support multiple users with complete data isolation and RBAC
Manage multiple AI models with model-specific memory scoping
Search semantically using vector embeddings with pgvector
Cache intelligently with Redis for performance
Scale horizontally in Kubernetes/RunPods environments

Architecture

┌─────────────────────────────────────────────────────┐
│              MemoryEngine (Orchestrator)            │
├─────────────────────────────────────────────────────┤
│                                                      │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────┐  │
│  │  User Mgmt   │  │ Embedding    │  │ Context  │  │
│  │ (RBAC)       │  │ Generation   │  │ Builder  │  │
│  └──────────────┘  └──────────────┘  └──────────┘  │
│                                                      │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────┐  │
│  │  Processor   │  │  Searcher    │  │ Models   │  │
│  │ (Ingestion)  │  │ (Retrieval)  │  │ Registry │  │
│  └──────────────┘  └──────────────┘  └──────────┘  │
│                                                      │
│  ┌─────────────────────────────────────────────┐   │
│  │     StoragePool (Connection Management)     │   │
│  │  ┌──────────────────┐  ┌──────────────────┐ │   │
│  │  │  PostgreSQL      │  │  Redis           │ │   │
│  │  │  (Data Layer)    │  │  (Cache Layer)   │ │   │
│  │  └──────────────────┘  └──────────────────┘ │   │
│  └─────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘

Features

Multi-User Support

User account management with roles (Admin, User, Service, Viewer)
Role-based access control (RBAC)
Complete data isolation between users
Audit logging for compliance

Memory Types

Conversations: Full chat history with messages
Facts: Learned information with confidence scoring
Images: Generated image metadata and parameters
Transcriptions: Audio transcription with language detection
Summaries: Session and context summaries

Vector Search

pgvector integration for semantic search
Embedding generation with ONNX Runtime
Configurable similarity thresholds
Full-text search fallback

Distributed Architecture

PostgreSQL for persistent storage
Redis for caching and session management
Connection pooling for efficiency
Health checks and automatic failover

Cloud-Ready

Docker Compose for local development
Kubernetes-ready configuration
RunPods compatible
Horizontally scalable

Installation

Prerequisites

Rust 1.70+
Docker & Docker Compose (for local development)
PostgreSQL 14+ with pgvector extension
Redis 6+

Quick Start

Clone the repository

git clone https://github.com/Mr-soloDev/Halldyll-Memory-Model
cd Halldyll-Memory-Model

Set up environment

cp .env.example .env
docker-compose up -d

Run migrations

sqlx migrate run --database-url postgresql://halldyll:halldyll_dev_password@localhost/halldyll_memory

Add to your Cargo.toml

[dependencies]
halldyll_memory_model = "0.2"

# Or from Git:
halldyll_memory_model = { git = "https://github.com/Mr-soloDev/Halldyll-Memory-Model" }

Usage

use halldyll_memory_model::{Config, MemoryEngine};
use halldyll_memory_model::models::{Conversation, Message, Role};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create and initialize engine
    let config = Config::default();
    let engine = MemoryEngine::new(config).await?;
    engine.initialize().await?;

    // Create a conversation
    let mut conversation = Conversation::new("My First Conversation");
    conversation.add_message(Message::new(Role::User, "Hello!"));
    conversation.add_message(Message::new(Role::Assistant, "Hi there!"));

    // Store memory
    let user_id = "user-123";
    engine.store_memory(user_id, Memory::Conversation(conversation)).await?;

    // Search memories
    let results = engine.search(user_id, "hello", 10).await?;
    println!("Found {} memories", results.len());

    Ok(())
}

Configuration

Environment Variables

# PostgreSQL
DATABASE_URL=postgresql://user:password@host:5432/db

# Redis
REDIS_URL=redis://host:6379

# Embeddings
EMBEDDING_MODEL_PATH=./models/all-MiniLM-L6-v2.onnx
EMBEDDING_DIMENSION=384

# Search
SEARCH_TOP_K=10
SIMILARITY_THRESHOLD=0.7

# Cache
CACHE_SIZE=1000

# TTL
EPHEMERAL_FACT_TTL_DAYS=30
MAX_CONTEXT_TOKENS=4000

# Database
DB_POOL_SIZE=5

# Logging
RUST_LOG=info,halldyll_memory_model=debug

Module Structure

src/
├── core/              # Fundamental types and errors
│   ├── config.rs      # Configuration
│   ├── error.rs       # Error types
│   └── id.rs          # ID generation
├── models/            # Data models
│   ├── conversation.rs
│   ├── fact.rs
│   ├── image.rs
│   ├── memory.rs
│   ├── transcription.rs
│   └── user_model.rs
├── user/              # User management
│   ├── manager.rs
│   └── permissions.rs
├── storage/           # Persistence layer
│   ├── pool.rs        # Connection pooling
│   ├── postgres.rs    # PostgreSQL operations
│   └── redis.rs       # Redis caching
├── embedding/         # Vector embeddings
├── ingest/            # Data ingestion
├── retrieval/         # Memory search
├── context/           # Prompt context building
├── engine/            # Main orchestrator
└── utils/             # Utility functions

Development

Run Tests

# Unit tests
cargo test --lib

# Integration tests (requires database)
cargo test --test '*' -- --test-threads=1

# All tests with logging
RUST_LOG=debug cargo test -- --nocapture

Build

# Debug
cargo build

# Release (optimized)
cargo build --release

Code Quality

# Format
cargo fmt

# Lint
cargo clippy -- -D warnings

# Check
cargo check

# Full validation
cargo check && cargo fmt && cargo clippy && cargo test

Generate Documentation

cargo doc --no-deps --open

Deployment

Docker

docker build -t halldyll-memory:latest .
docker run -e DATABASE_URL=postgresql://... \
           -e REDIS_URL=redis://... \
           halldyll-memory:latest

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: halldyll-memory
spec:
  replicas: 2
  selector:
    matchLabels:
      app: halldyll-memory
  template:
    metadata:
      labels:
        app: halldyll-memory
    spec:
      containers:
      - name: memory
        image: halldyll-memory:latest
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: url
        - name: REDIS_URL
          valueFrom:
            secretKeyRef:
              name: redis-credentials
              key: url

RunPods

Compatible with RunPods serverless and pod deployments. Ensure PostgreSQL and Redis are accessible.

API Overview

Memory Storage

pub async fn store_memory(
    &self,
    user_id: &str,
    memory: Memory,
) -> MemoryResult<()>

Search

pub async fn search(
    &self,
    user_id: &str,
    query: &str,
    limit: usize,
) -> MemoryResult<Vec<Memory>>

User Management

pub async fn create_user(
    &self,
    username: String,
    email: String,
    role: UserRole,
) -> MemoryResult<UserModel>

Performance

Vector search: < 100ms for top-10 on 1M vectors (with proper indexing)
Fact storage: Non-blocking, fully asynchronous
Embedding caching: Configurable LRU cache
Database: Connection pooling with configurable pool size

Security

User Isolation: Complete data separation per user
RBAC: Role-based access control with permission checking
Audit Logging: All operations logged for compliance
No SQL Injection: Uses parameterized queries (sqlx)
Validation: Input validation for all user data

Testing Models

Supported model specialties:

Qwen2.5-14B-Instruct: LLM for text generation
Qwen2.5-VL-7B-Instruct: Vision-language for image understanding
RealVisXL_V5.0: Ultra-realistic image generation
ControlNet/InstantID: Image generation control and identity preservation
InsightFace: Facial recognition

Roadmap

GraphQL API layer
WebSocket support for real-time memory updates
Advanced fact extraction with LLM
Multi-language embedding support
Hierarchical memory compression
Import/Export functionality
Web dashboard for memory visualization
Model-specific memory scoping
Distributed tracing with OTEL

Contributing

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit changes (git commit -m 'Add amazing feature')
Push to branch (git push origin feature/amazing-feature)
Open a Pull Request

License

MIT License - see LICENSE file for details

Author

Roy Géryan - Halldyll Creator

Support

For issues, questions, or suggestions:

GitHub Issues: https://github.com/Mr-soloDev/Halldyll-Memory-Model/issues
Discussions: https://github.com/Mr-soloDev/Halldyll-Memory-Model/discussions
Email: geryan.roy@icloud.com

Built with ❤️ for Halldyll and the AI community

halldyll_memory_model 0.2.0