reddb-io 1.23.1

Unified multi-model database: tables, documents, graphs, vectors, and key-value in one engine. Umbrella crate that hosts the `red` binary and re-exports the workspace.
Documentation

🎯 Why RedDB?

Stop running Postgres + Neo4j + Pinecone + Redis + Mongo + InfluxDB + RabbitMQ. One file. One engine. One query language. All seven models work together seamlessly.

[!IMPORTANT] Collections: In RedDB, a collection is the named logical container for data. Tables, documents, key-value, graphs, vectors, time-series, and queues are the models you layer on top. users can be a table, events can be documents, config can be KV — in the same collection or spread across many. Choose your model; collections unite them.

✨ The Killer Feature: ASK

ASK 'who owns passport AB1234567 and what services do they use?'

One command. RedDB searches across tables, graphs, vectors, documents, and key-value stores — builds context — calls an LLM — returns a natural-language answer. No pipelines. No glue code. No other database does this.

📊 7 Data Models, 1 Engine

The mental model: your model is how you query (table, graph, vector, etc.), your collection is where the data lives. Mix and match freely.

-- 📋 Relational rows
INSERT INTO users (name, email) VALUES ('Alice', 'alice@co.com')

-- 📄 JSON documents — inline JSON syntax
INSERT INTO logs DOCUMENT VALUES ({"level": "info", "msg": "login"})

-- 🔗 Graph edges  
INSERT INTO network EDGE (label, from_rid, to_rid) VALUES ('CONNECTS', 102, 103)

-- 🎯 Vector similarity search
SEARCH SIMILAR TEXT 'anomaly detected' COLLECTION events

-- 🔑 Key-value
KV PUT config.theme = 'dark'

-- ⏱️  Time-series (retention & downsampling)
CREATE TIMESERIES cpu_metrics RETENTION 90 d
INSERT INTO cpu_metrics (metric, value, tags) VALUES ('cpu.idle', 95.2, {"host": "srv1"})

-- 📦 Hypertables (logs, events, telemetry)
CREATE HYPERTABLE access_log TIME_COLUMN ts CHUNK_INTERVAL '1d' TTL '90d'

-- 📑 Append-only (audit, ledger, immutable)
CREATE TABLE audit_log (id BIGINT, action TEXT) APPEND ONLY

-- 📨 Message queues (FIFO, priority, consumer groups)
CREATE QUEUE tasks MAX_SIZE 10000
QUEUE PUSH tasks {"job": "process", "id": 123}
QUEUE POP tasks

Same file. Same engine. Same query language — everything lives in one .rdb file.

Want to use RedDB as your log store? Start with the Logs Quickstart or the full Using RedDB for Logs guide.

🧠 AI-Native From Day One

-- Semantic search without managing vectors yourself
SEARCH SIMILAR TEXT 'suspicious login' COLLECTION logs USING groq

-- Auto-embed on insert — vectors are created automatically
INSERT INTO articles (title, body) VALUES ('AI Safety', 'Alignment research...')
  WITH AUTO EMBED (body) USING openai

-- Context search across all data models
SEARCH CONTEXT '192.168.1.1' FIELD ip DEPTH 2

-- Ask questions in plain English, get grounded answers
ASK 'what vulnerabilities affect host 10.0.0.1?' USING anthropic

RAG built into the database layer. RedDB retrieves context from every data model and feeds it to the LLM.

🤖 11 AI Providers

Swap providers with a keyword. No code changes. Full parity across ASK, embeddings, and model selection.

Provider Keyword API Key ASK / Prompt Embeddings
OpenAI openai ✅ ✅ ✅
Anthropic anthropic ✅ ✅ —*
Groq groq ✅ ✅ ✅
OpenRouter openrouter ✅ ✅ ✅
Together together ✅ ✅ ✅
Venice venice ✅ ✅ ✅
DeepSeek deepseek ✅ ✅ ✅
HuggingFace huggingface ✅ ✅ ✅
Ollama ollama — ✅ ✅
Local local — feature-gated feature-gated
Custom URL https://... varies ✅ ✅

*Anthropic does not offer embeddings; RedDB rejects embedding calls explicitly rather than re-routing to another provider.

Most providers speak the OpenAI-compatible POST /embeddings shape; HuggingFace has its own (POST /pipeline/feature-extraction/{model}) and RedDB ships a dedicated client for it. Anthropic does not have an embeddings API — RedDB rejects embedding calls against it explicitly rather than silently re-routing to a different provider. local requires the local-models feature flag at engine build time.

See docs/guides/ai-providers.md for the routing matrix, the wire shape per provider, and the Anthropic-embeddings policy in detail.

ASK 'summarize alerts' USING groq MODEL 'llama-3.3-70b-versatile'
ASK 'summarize alerts' USING ollama MODEL 'llama3'
ASK 'summarize alerts' USING anthropic

Set a default provider so you can drop USING from every query:

# Set default provider -- no more USING on every query
curl -X POST http://127.0.0.1:5000/ai/credentials \
  -d '{"provider":"groq","api_key":"gsk_xxx","default":true}'

-- Now ASK uses groq by default
ASK 'what happened?'

# Export/import all config as JSON
curl http://127.0.0.1:5000/config

🎲 Probabilistic Data Structures

Built-in approximate data structures for real-time analytics at scale. Cardinality estimation, frequency analysis, and set membership all in one engine.

-- HyperLogLog: unique visitor count (~0.8% error, 16KB memory)
CREATE HLL visitors
HLL ADD visitors 'user1' 'user2' 'user3'
HLL COUNT visitors

-- Count-Min Sketch: frequency estimation
CREATE SKETCH click_counter WIDTH 2000 DEPTH 7
SKETCH ADD click_counter 'button_a' 5
SKETCH COUNT click_counter 'button_a'

-- Cuckoo Filter: membership + deletion (unlike Bloom)
CREATE FILTER active_sessions CAPACITY 500000
FILTER ADD active_sessions 'session_abc'
FILTER CHECK active_sessions 'session_abc'
FILTER DELETE active_sessions 'session_abc'

🗂️ Advanced Indexes

Beyond B-tree. Pick the right index for your access pattern.

-- Hash: O(1) exact-match lookups
CREATE INDEX idx_email ON users (email) USING HASH

-- Bitmap: analytical queries on low-cardinality columns
CREATE INDEX idx_status ON orders (status) USING BITMAP

-- R-Tree: spatial & geographic queries
CREATE INDEX idx_loc ON sites (location) USING RTREE
SEARCH SPATIAL RADIUS 48.8566 2.3522 10.0 COLLECTION sites COLUMN location LIMIT 50
SEARCH SPATIAL NEAREST 48.8566 2.3522 K 5 COLLECTION sites COLUMN location

🔧 SQL Extensions

RedDB extends SQL with WITH clauses for operational semantics:

-- TTL: auto-expire records
INSERT INTO sessions (token) VALUES ('abc') WITH TTL 1 h

-- Context indexes for cross-model search
CREATE TABLE customers (passport TEXT) WITH CONTEXT INDEX ON (passport)

-- Graph expansion inline with SELECT
SELECT * FROM users WITH EXPAND GRAPH DEPTH 2

-- Metadata on write
INSERT INTO logs (msg) VALUES ('deploy') WITH METADATA (source = 'ci')

-- Absolute expiration
INSERT INTO events (name) VALUES ('launch') WITH EXPIRES AT 1735689600000

🔤 6 Query Languages

Write in whatever you think in. The engine auto-detects the language.

Language Example
SQL SELECT * FROM hosts WHERE os = 'linux'
Cypher MATCH (a:User)-[:FOLLOWS]->(b) RETURN b.name
Gremlin g.V().hasLabel('person').out('FOLLOWS').values('name')
SPARQL SELECT ?name WHERE { ?p :name ?name }
Natural Language show me all critical hosts
ASK (RAG) ASK 'what changed in the last 24 hours?'

All six hit the same engine, same data, same indexes.

🔄 Native Migrations — No External Tools

Stop reaching for Flyway, Liquibase, Drizzle Migrate, or Sequelize. RedDB handles schema and data migrations as first-class SQL commands — all in one transaction log.

-- Register a migration
CREATE MIGRATION add_users_table AS
  CREATE TABLE users (id BIGINT, email TEXT, created_at TIMESTAMP);

-- Register a dependent migration (RedDB also auto-infers deps from SQL body)
CREATE MIGRATION add_users_index DEPENDS ON add_users_table AS
  CREATE INDEX idx_email ON users (email);

-- Apply everything in dependency order
APPLY MIGRATION *

-- Large data backfill in safe 5,000-row batches — resumes on crash
CREATE MIGRATION backfill_display_names BATCH 5000 ROWS AS
  UPDATE users SET display_name = email WHERE display_name IS NULL;

-- Undo an applied migration (VCS revert under the hood)
ROLLBACK MIGRATION add_users_index

-- Inspect what a migration will do
EXPLAIN MIGRATION backfill_display_names

Every applied migration creates a VCS commit (RedDB's "Git for Data"). Rollback reverts that commit automatically — no rollback scripts to maintain. Dependency ordering is a DAG; RedDB detects cycles at CREATE time and auto-infers edges from your SQL body so you rarely need explicit DEPENDS ON.

→ Native Migrations docs

📦 48 Built-in Types

Not just TEXT and INTEGER. RedDB understands your domain — validation on write, no parsing in your app.

Category Types
Network IpAddr, Ipv4, Ipv6, MacAddr, Cidr, Subnet, Port
Geographic Latitude, Longitude, GeoPoint
Locale Country2, Country3, Lang2, Lang5, Currency
Identity Uuid, Email, Url, Phone, Semver
Visual Color, ColorAlpha
Cross-model Refs NodeRef, EdgeRef, VectorRef, RowRef, KeyRef, DocRef, TableRef, PageRef
Primitives Integer, UnsignedInteger, Float, Decimal, BigInt, Text, Blob, Boolean, Json, Array, Enum
Temporal Timestamp, TimestampMs, Date, Time, Duration

💾 Backup & Recovery

Built-in backup scheduler, WAL archiving, CDC, and Point-in-Time Recovery — no sidecars required:

# Poll real-time changes
curl 'localhost:5000/changes?since_lsn=0'

# Trigger manual backup
curl -X POST localhost:5000/backup/trigger

# Check backup status
curl localhost:5000/backup/status

Remote backends: S3, R2, DigitalOcean Spaces, GCS, Turso, Cloudflare D1, local filesystem.

For concrete RTO/RPO numbers per failure mode (process crash, disk loss, PITR rollback, replica promotion), see docs/operations/rto-rpo.md.

🔌 KV REST API

Every collection doubles as a key-value store with dedicated REST endpoints:

# Write a key
curl -X PUT http://127.0.0.1:5000/collections/settings/kvs/theme \
  -H 'content-type: application/json' -d '{"value": "dark"}'

# Read a key
curl http://127.0.0.1:5000/collections/settings/kvs/theme

# Delete a key
curl -X DELETE http://127.0.0.1:5000/collections/settings/kvs/theme

Config keys work the same way -- read, write, or delete any red_config setting at runtime:

# Set a config key
curl -X PUT http://127.0.0.1:5000/config/red.ai.default.provider \
  -d '{"value": "groq"}'

# Read a config key
curl http://127.0.0.1:5000/config/red.ai.default.provider

# Or manage config from SQL
SET CONFIG red.ai.default.provider = 'groq'
SHOW CONFIG red.ai

🌍 3 Deployment Modes

Mode Like Access Use Case
Embedded SQLite Rust API (RedDB::open("data.rdb")) In-process, single machine
Server Postgres RedWire + gRPC + HTTP Multi-client, networked
Agent MCP Server red mcp Claude Code / AI agents

Same storage format. Start embedded, scale to server, expose to agents — zero migration.

⚡ Performance

Where RedDB wins: benchmarks show measurable wins over Postgres and Mongo today:

  • typed_insert — 16× faster than PostgreSQL on typed single-row inserts
  • disk_usage — 1.5× faster than MongoDB on compact-write workloads

See docs/perf/wins.md for reproducible benchmarks and docs/perf/when-not-reddb.md for the honest gaps where we're still behind.

RedDB uses multiple optimization techniques for fast queries at scale:

  • Result Cache -- identical SELECT queries return in <1ms; auto-invalidated on INSERT/UPDATE/DELETE (30s TTL, max 1000 entries)
  • Hot Item Cache -- get_any(rid) lookups served from an LRU cache (10K entries), O(1) instead of scanning all collections
  • Binary Bulk Insert -- gRPC BulkInsertBinary with zero JSON overhead, protobuf native types -- 241K ops/sec
  • Concurrent HTTP -- thread-per-connection model; each request handled in its own OS thread
  • Parallel Segment Scanning -- sealed segments scanned in parallel via std::thread::scope; auto-detects single-core and skips parallelism
  • Hash Join -- O(n+m) joins instead of O(n*m), auto-selected for large datasets
  • Lazy Graph Materialization -- only loads reachable nodes instead of full graph
  • Pre-filtered Vector Search -- metadata filters applied before HNSW indexing
  • Index-Assisted Scans -- bloom filter + hash index hints for WHERE clauses
  • Column Projection Pushdown -- only materializes SELECT columns
  • Query Plan Caching -- LRU cache with 1h TTL for repeated queries
  • Batch Entity Lookup -- multi-entity fetches resolved in a single pass
  • Background Maintenance Thread -- backup scheduling, retention, and checkpoint run off the hot path

🛡️ Durability & Corruption Defense

Seven layers of protection — tested and proven against power loss, torn writes, and crashes:

Layer Defense
File Lock Exclusive flock — prevents concurrent writes to .rdb
Double-Write Buffer Pages in .rdb-dwb first — survives power loss
Header Shadow Backup page 0 in .rdb-hdr — auto-recovers corruption
Metadata Shadow Backup page 1 in .rdb-meta — recovers collection registry
fsync Discipline sync_all() after every critical write — no flush-only shortcuts
Two-Phase Checkpoint WAL→DB with checkpoint_in_progress flag — crash-safe
CRC32 Checksum Every page, every WAL record, full-file footer — detects bitrot

🔀 Eventual Consistency

RedDB supports per-field eventual consistency via an append-only transaction log with periodic consolidation. Inspired by CRDT principles (commutative, associative reducers), it enables high-throughput write patterns while guaranteeing convergence.

# Track clicks with async consolidation (returns instantly)
curl -X POST localhost:5000/ec/urls/clicks/add -d '{"id": 1, "value": 1}'

# Check consolidated + pending value
curl localhost:5000/ec/urls/clicks/status?id=1
Feature Description
6 reducers Sum, Max, Min, Count, Average, Last (last-write-wins)
Sync mode Consolidates immediately (strong consistency)
Async mode Background worker consolidates periodically (high throughput)
Transaction log Immutable append-only audit trail per field
SET checkpoint Resets base value, discards prior operations
All modes Works in server, embedded (Rust API), and serverless

See the Eventual Consistency Guide for the theory (CAP theorem, CRDTs, convergence) and full API reference.

🗺️ Geographic Operations

Built-in geo functions with no external dependencies. Supports both spherical (Haversine) and ellipsoidal (Vincenty/WGS-84) models.

-- Distance from each store to a point (in km)
SELECT name, GEO_DISTANCE(location, POINT(-23.55, -46.63)) AS dist
FROM stores ORDER BY dist

-- Vincenty for sub-millimeter accuracy
SELECT name, GEO_DISTANCE_VINCENTY(location, POINT(40.71, -74.00)) AS dist
FROM airports

# HTTP API
curl -X POST localhost:5000/geo/distance -d '{
  "from": {"lat": -23.55, "lon": -46.63},
  "to": {"lat": -22.91, "lon": -43.17}
}'
Function What it computes
GEO_DISTANCE Haversine distance (km)
GEO_DISTANCE_VINCENTY WGS-84 geodesic distance (km)
GEO_BEARING Compass direction (degrees)
GEO_MIDPOINT Great-circle midpoint

Also available: destination point, bounding box, polygon area, spatial search (RADIUS, BBOX, NEAREST). See the Geo Operations Guide.

📈 Vector Clustering

Standalone K-Means and DBSCAN clustering on vector collections, with SIMD-accelerated distance computation and automatic parallelization.

# K-Means: group products into 5 clusters
curl -X POST localhost:5000/vectors/cluster -d '{
  "collection": "products", "algorithm": "kmeans", "k": 5
}'

# DBSCAN: discover clusters automatically (no K needed)
curl -X POST localhost:5000/vectors/cluster -d '{
  "collection": "products", "algorithm": "dbscan", "eps": 0.5, "min_points": 3
}'

K-Means uses parallel assignment (multi-threaded for datasets > 1K vectors). DBSCAN labels unreachable points as noise (-1), useful for outlier detection. See the Vector Clustering Guide.

🚀 Native Drivers

One connection-string API, multiple languages. Same connect(uri) everywhere — code ports across runtimes with zero ceremony.

Language Package Install Backends
Rust reddb-io-client cargo add reddb-io-client embedded ✅ · gRPC ✅ · HTTP ✅
Node / Bun / Deno @reddb-io/sdk (npm) pnpm add @reddb-io/sdk stdio subprocess ✅
Python reddb (PyPI) pip install reddb (soon) embedded ✅ · gRPC ✅ · wire ✅

All drivers accept the same URIs:

memory://                   ephemeral in-memory
file:///absolute/path       embedded engine on disk
grpc://host:port            remote server (planned — tracked in PLAN_DRIVERS.md)

Example — the same app in three languages:

// Rust
let db = reddb_client::Reddb::connect("memory://").await?;
db.insert("users", &JsonValue::object([("name", JsonValue::string("Alice"))])).await?;
let rows = db.query("SELECT * FROM users").await?;

// Node, Bun, Deno
import { connect } from '@reddb-io/sdk'
const db = await connect('memory://')
await db.insert('users', { name: 'Alice' })
const rows = await db.query('SELECT * FROM users')

# Python
import reddb
with reddb.connect("memory://") as db:
    db.insert("users", {"name": "Alice"})
    print(db.query("SELECT * FROM users"))

Driver docs live in crates/reddb-client/README.md, drivers/js/README.md, and drivers/python/README.md. The full protocol spec and roadmap are in PLAN_DRIVERS.md.

For JavaScript and TypeScript, RedDB ships three packages under the @reddb-io/ scope. Pick the one that matches your scenario — see the JavaScript / TypeScript driver guide for the full matrix and ADR 0007 for the rationale.

# App code in Node, Bun, or Deno — full SDK with embedded, gRPC, and HTTP transports
pnpm add @reddb-io/sdk

# Thin remote-only client for serverless, edge, CI, or sidecar runtimes (~5 MB)
pnpm add @reddb-io/client

# CLI launcher — installs the `red` binary on PATH
pnpm add -g @reddb-io/cli

Application code with the SDK:

import { connect } from '@reddb-io/sdk'

const db = await connect('memory://')
const result = await db.query('SELECT * FROM users')
await db.close()

Launch the server from npm without a separate install step:

npx @reddb-io/cli@latest version
npx @reddb-io/cli@latest server --wire-bind 127.0.0.1:5050 --http-bind 127.0.0.1:5000 --path ./data.rdb

🚀 Quick Start

Install the red binary:

curl -fsSL https://raw.githubusercontent.com/reddb-io/reddb/main/install.sh | bash
# Verifies SHA256 before install • auto-detects your OS + arch

Start the server:

red server --wire-bind 127.0.0.1:5050 --http-bind 127.0.0.1:5000 --path ./data.rdb
# RedWire: 5050 · gRPC: 55055 · HTTP: 5000

Insert and query via HTTP:

curl -X POST http://127.0.0.1:5000/query \
  -H 'content-type: application/json' \
  -d '{"query":"INSERT INTO hosts (ip, os) VALUES ('"'"'10.0.0.1'"'"', '"'"'linux'"'"')"}'

curl -X POST http://127.0.0.1:5000/query \
  -H 'content-type: application/json' \
  -d '{"query":"SELECT * FROM hosts"}'

Or via npm (no separate install):

npx @reddb-io/cli@latest server --http-bind 127.0.0.1:5000

Or Docker:

docker run --rm -p 5050:5050 -p 55055:55055 -p 5000:5000 \
  ghcr.io/reddb-io/reddb:latest

→ See docs/deployment/docker.md and docs/security/vault.md for production setups.

📂 Architecture

RedDB ships as a Cargo workspace:

Crate Role
reddb Binary umbrella; houses the red CLI
reddb-io-engine Storage engine, indexes, query execution
reddb-io-client Rust driver with embedded, gRPC, HTTP transports
reddb-io-wire RedWire protocol vocabulary & framing
reddb-io-rql RQL parser and semantic analyzer
reddb-io-types Type system, value codec, cross-model refs

📚 Resources

Resource Purpose
📖 Documentation Full guides, API reference, tutorials
🐙 GitHub Source code, issues, discussions
📦 npm CLI Install & run via npx
📤 npm Drivers Node/Bun/Deno SDK
📋 Release Notes Version history & changelog
🛡️ Security Policy Reporting vulnerabilities
✅ Contract Matrix Every promise → test or plan

Business Source License 1.1 — source-available; free for self-hosted & internal use. Managed services require a commercial license. Converts to AGPL-3.0 on June 22, 2030. See LICENSE.

Built by RedDB.io

Public-surface support

Generated from docs/conformance/public-surface-contract-matrix.json by scripts/gen-docs-from-matrix.mjs. Do not edit between the markers by hand — run node scripts/gen-docs-from-matrix.mjs --write. The matrix is the source of truth; this block can never claim more than it, and CI (docs-matrix) fails on drift.

Every public RedDB promise and the status of each public surface that offers it.

Promise sql http redwire grpc driver_helpers
PSC-001 — RedDB is one multi-model database (tables, graph, KV, timeseries, probabilistic, vector, queue, documents) backed by a single file. ✅ supported ✅ supported ✅ supported ✅ supported ✅ supported
PSC-002 — MATCH supports node, edge, label, property, and LIMIT projections. ✅ supported ✅ supported ⚠️ partial ⚠️ partial ✅ supported
PSC-003 — GRAPH algorithms accept semantic identifiers, limits, ordering, and return stable rich rows. ✅ supported ✅ supported ❌ unsupported ❌ unsupported ❌ unsupported
PSC-004 — INSERT creates rows, documents, and native timeseries points. ✅ supported ✅ supported ⚠️ partial ✅ supported ✅ supported
PSC-005 — HLL/SKETCH/FILTER expose write and read commands for cardinality, frequency, and membership. ⚠️ partial ❌ unsupported ❌ unsupported ❌ unsupported ⚠️ partial
PSC-006 — Timeseries stores timestamped metrics with tags and supports query/readback. ✅ supported ⚠️ partial ❌ unsupported ❌ unsupported ⚠️ partial
PSC-007 — Documents are first-class: create, read, update, delete, and SQL analytics over JSON. ✅ supported ✅ supported ❌ unsupported ✅ supported ✅ supported
PSC-008 — KV helpers expose get/put/delete; get of a missing key returns null, delete reports affected. ✅ supported ❌ unsupported ❌ unsupported ✅ supported ✅ supported
PSC-009 — Queue helpers expose create/push/peek/pop/len/purge with FIFO semantics; empty pop is not an error. ✅ supported ❌ unsupported ❌ unsupported ❌ unsupported ✅ supported
PSC-010 — Transactions are imperative (begin/commit/rollback) plus a run(callback) form; empty SQL rejects with INVALID_ARGUMENT. ✅ supported ❌ unsupported ❌ unsupported ✅ supported ✅ supported
PSC-011 — SQL aggregate, projection, expression, and mutation behaviour matches ordinary SQL expectations where advertised. ✅ supported ✅ supported ⚠️ partial ⚠️ partial ✅ supported
PSC-012 — Server transports expose the same query contract as embedded (HTTP, RedWire, gRPC parity). ✅ supported ✅ supported ✅ supported ✅ supported ✅ supported
PSC-013 — Official drivers implement the SDK Helper Spec v1.0 conformance suite (all 22 §12 case IDs). ❌ unsupported ❌ unsupported ❌ unsupported ✅ supported ✅ supported
PSC-014 — ASK / SEARCH semantic surfaces return ranked results with stable shape. ✅ supported ⚠️ partial ❌ unsupported ❌ unsupported ⚠️ partial

Status legend: ✅ supported · ⚠️ partial (known gaps) · ❌ unsupported.