MagnumDB

Modern Open Source Embedded Database Engine in Native Rust

MagnumDB is an open-source embedded key-value and SQL database engine written 100% from scratch in native Rust. Designed for performance, reliability, and modularity.

Features

Embedded Engine: Runs directly inside Rust binaries with zero external C/C++ dependencies.
WAL Durability: Write-Ahead Logging (WAL) with TxID framing and CRC32 checksums ensures crash recovery.
B+ Tree Indexing: Custom 4KB page disk pager, LRU buffer pool management, overflow pages, and leaf page recycling.
Relational SQL Query Engine: Hash JOINs (INNER JOIN, LEFT JOIN), GROUP BY & HAVING analytical aggregations, range filters, and composite multi-column indexes.
MVCC & Transactions: MVCC row headers (xmin, xmax) and transaction logging with BEGIN, COMMIT, and ROLLBACK.
PostgreSQL Protocol Compatibility: Native PostgreSQL Wire Protocol (pgwire) handling StartupMessage, RowDescription (T), DataRow (D), and CommandComplete (C).
Multi-Client TCP Server: Async TCP server powered by Tokio with connection limits and idle timeouts.

What's New in v0.3.0

Version 0.3.0 is a milestone release introducing relational query capabilities and PostgreSQL protocol compatibility:

🔗 Relational JOINs: Support for INNER JOIN and LEFT JOIN using streaming HashJoinExec Volcano operators:
```
SELECT users.name, orders.amount FROM users JOIN orders ON users.id = orders.user_id;
```
📊 GROUP BY & HAVING Aggregations: Streaming HashGroupAggregateExec operator for analytical groupings:
```
SELECT dept, COUNT(*) FROM employees GROUP BY dept HAVING COUNT(*) > 1;
```
📑 Composite Multi-Column Indexes: Supports secondary indexing over multiple columns:
```
CREATE INDEX idx_name ON users(last_name, first_name);
```
🐘 Native PostgreSQL Wire Protocol (pgwire): Server auto-detects and serves native PostgreSQL protocol connections (StartupMessage, RowDescription, DataRow, CommandComplete, ReadyForQuery).
⏳ MVCC Tuple Headers: Binary row encoding incorporates [xmin: 8B][xmax: 8B] transaction visibility metadata.

Architecture

graph TD
    A[Client / magnum shell / psql] -->|SQL Query| B(SQL Parser)
    B -->|AST| C(Query Executor)
    
    C -->|Reads/Writes| D[B+ Tree Index]
    C -->|Logs| E[(WAL - Write Ahead Log)]
    
    D -->|Request Page| F(Buffer Pool Manager)
    F -->|Evict/Load 4KB Pages| G[(Disk / Pager)]
    
    style A fill:#f9f,stroke:#333,stroke-width:2px
    style D fill:#bbf,stroke:#333,stroke-width:2px
    style F fill:#bbf,stroke:#333,stroke-width:2px
    style G fill:#dfd,stroke:#333,stroke-width:2px
    style E fill:#dfd,stroke:#333,stroke-width:2px

Installation

Add MagnumDB to your Cargo.toml:

[dependencies]
magnumdb = "0.3.0"

Quick Start (Embedded Key-Value)

use magnumdb::{Database, Config};

fn main() -> anyhow::Result<()> {
    let config = Config::default().with_path("./my_database");
    let mut db = Database::open(config)?;

    // Embedded Key-Value API
    db.put(b"user:100", b"Soham")?;
    let val = db.get(b"user:100")?;
    
    if let Some(bytes) = val {
        println!("Found: {}", String::from_utf8_lossy(&bytes));
    }

    db.close()?;
    Ok(())
}

Embedded SQL Usage

use magnumdb::{Database, Config};
use magnumdb::sql::{Executor, Parser};

fn main() -> anyhow::Result<()> {
    let config = Config::default().with_path("./sql_data");
    let mut db = Database::open(config)?;
    let mut exec = Executor::new(&mut db);

    exec.execute(Parser::parse("CREATE TABLE users(id INT, name TEXT)")?)?;
    exec.execute(Parser::parse("INSERT INTO users VALUES(1, 'Alice')")?)?;
    
    let res = exec.execute(Parser::parse("SELECT * FROM users")?)?;
    println!("{}", res);

    Ok(())
}

Contributing & License

We welcome contributions! Please review CONTRIBUTING.md.

Licensed under the MIT License.

magnumdb 0.3.0