MagnumDB

Modern Open Source Embedded Database Engine in Native Rust

MagnumDB is an open-source embedded key-value and SQL database engine written 100% from scratch in native Rust. Designed for performance, reliability, and modularity.

Features

Embedded Engine: Runs directly inside Rust binaries with zero external C/C++ dependencies.
WAL Durability: Write-Ahead Logging (WAL) with TxID framing and CRC32 checksums ensures crash recovery.
B+ Tree Indexing: Custom 4KB page disk pager, LRU buffer pool management, overflow pages, and leaf page recycling.
SQL Execution Engine: Built-in SQL AST parser, Volcano-style streaming query execution, range filters, and secondary indexes.
Multi-Client TCP Server: Async TCP server powered by Tokio with connection limits and idle timeouts.
ACID Transactions: Transaction logging with BEGIN, COMMIT, and ROLLBACK support.

What's New in v0.2.0

Version 0.2.0 represents a major production stability release resolving 14 core database engine issues:

🛡️ Durable Transactions: Transaction writes properly track tx_id in WAL records, and committed pages are flushed and synced on COMMIT.
🔒 Page Serialization Protection: B+ Tree page serialization includes bounds-checking guards against 4KB overflows.
📐 Documented Metadata Page Layout: Page 0 layout standardized with magic bytes (MGDB), root page ID, free-list head, and checkpoint LSN.
⚡ Optimized Crash Recovery: Startup WAL recovery uses checkpoint LSN filtering to avoid full file replay.
🧹 Secondary Index Maintenance: DELETE and UPDATE SQL queries automatically update and clean up secondary indexes.
👥 Multi-PK Secondary Indexes: Secondary indexes now support multiple primary keys per indexed value (1:N mapping).
💬 Enhanced SQL Parsing: Supports string literals with spaces, commas ('hello, world'), and escaped quotes ('').
🛑 Identifier Validation: Reserved system namespace (__) protects catalog tables from SQL injection.
🌐 Async TCP Server Hardening: Connection semaphore limits (max_connections) and idle read timeouts.

Architecture

graph TD
    A[Client / magnum shell] -->|SQL Query| B(SQL Parser)
    B -->|AST| C(Query Executor)
    
    C -->|Reads/Writes| D[B+ Tree Index]
    C -->|Logs| E[(WAL - Write Ahead Log)]
    
    D -->|Request Page| F(Buffer Pool Manager)
    F -->|Evict/Load 4KB Pages| G[(Disk / Pager)]
    
    style A fill:#f9f,stroke:#333,stroke-width:2px
    style D fill:#bbf,stroke:#333,stroke-width:2px
    style F fill:#bbf,stroke:#333,stroke-width:2px
    style G fill:#dfd,stroke:#333,stroke-width:2px
    style E fill:#dfd,stroke:#333,stroke-width:2px

Installation

Add MagnumDB to your Cargo.toml:

[dependencies]
magnumdb = "0.2.0"

Quick Start (Embedded Key-Value)

use magnumdb::{Database, Config};

fn main() -> anyhow::Result<()> {
    let config = Config::default().with_path("./my_database");
    let mut db = Database::open(config)?;

    // Embedded Key-Value API
    db.put(b"user:100", b"Soham")?;
    let val = db.get(b"user:100")?;
    
    if let Some(bytes) = val {
        println!("Found: {}", String::from_utf8_lossy(&bytes));
    }

    db.close()?;
    Ok(())
}

Embedded SQL Usage

use magnumdb::{Database, Config};
use magnumdb::sql::{Executor, Parser};

fn main() -> anyhow::Result<()> {
    let config = Config::default().with_path("./sql_data");
    let mut db = Database::open(config)?;
    let mut exec = Executor::new(&mut db);

    exec.execute(Parser::parse("CREATE TABLE users(id INT, name TEXT)")?)?;
    exec.execute(Parser::parse("INSERT INTO users VALUES(1, 'Alice')")?)?;
    
    let res = exec.execute(Parser::parse("SELECT * FROM users")?)?;
    println!("{}", res);

    Ok(())
}

Publishing & Updating Crates.io

To update or publish new versions to crates.io:

Login to crates.io via Cargo token:
```
cargo login <YOUR_CRATES_IO_TOKEN>
```
Package and verify dry-run build:
```
cargo package
```
Publish to Crates.io:
```
cargo publish
```

Contributing & License

We welcome contributions! Please review CONTRIBUTING.md.

Licensed under the MIT License.

magnumdb 0.2.0