Graph_D - Native Graph Database in Rust

A high-performance, memory-efficient native graph database implementation in Rust with built-in JSON support and ACID compliance.

🎯 Vision

Build a production-ready, memory-efficient native graph database that leverages Rust's safety guarantees and performance characteristics to provide:

Native storage with index-free adjacency for O(1) traversal performance
First-class JSON support for flexible node and relationship properties
Memory efficiency comparable to SQLite for embedded applications
Thread-safe, ACID-compliant operations with minimal overhead

✨ Features

Core Database Features

🚀 High Performance: 140K+ nodes/sec creation, 32M+ lookups/sec
💾 Memory Efficient: ~1.9KB per node including relationships and indexes
🔄 ACID Compliance: Full transaction support with multiple isolation levels
🔒 Concurrent Safe: Advanced locking with deadlock detection
📁 Persistent Storage: Memory-mapped files for durability

Query & Analytics

🔍 Advanced Queries: Traversal, filtering, aggregation, sorting
📊 Rich Aggregations: Count, sum, avg, min, max, group by, statistics
📄 Pagination: Efficient offset/limit support with sorting
🎯 Path Finding: BFS shortest path algorithms

Developer Experience

🦀 Rust Native: Zero-cost abstractions and memory safety
📖 Rich Documentation: Comprehensive examples and API docs
🧪 Well Tested: 32+ tests covering all major functionality
⚡ Async Ready: Built with tokio for async/await support

🚀 Quick Start

Installation

Add to your Cargo.toml:

[dependencies]
graph_d = "0.1.0"

Basic Usage

use graph_d::{Graph, Result};
use serde_json::json;

fn main() -> Result<()> {
    // Create a new in-memory graph
    let mut graph = Graph::new()?;
    
    // Create nodes with JSON properties
    let alice_id = graph.create_node([
        ("name".to_string(), json!("Alice")),
        ("age".to_string(), json!(30)),
        ("role".to_string(), json!("Engineer")),
    ].into())?;
    
    let bob_id = graph.create_node([
        ("name".to_string(), json!("Bob")),
        ("age".to_string(), json!(25)),
        ("role".to_string(), json!("Designer")),
    ].into())?;
    
    // Create relationships
    let rel_id = graph.create_relationship(
        alice_id,
        bob_id,
        "WORKS_WITH".to_string(),
        [("since".to_string(), json!("2023"))].into(),
    )?;
    
    // Query the graph
    if let Some(alice) = graph.get_node(alice_id)? {
        println!("Alice: {:?}", alice.properties);
    }
    
    // Find Alice's relationships
    let relationships = graph.get_relationships_for_node(alice_id)?;
    println!("Alice has {} relationships", relationships.len());
    
    Ok(())
}

💻 Command Line Interface

Graph_D also provides a standalone CLI binary for interactive database exploration, similar to sqlite3.

CLI Installation

# Install from crates.io (includes CLI)
cargo install graph_d --features cli

# Or build from source
cargo build --release --features cli

Interactive Shell

# Start with in-memory database
graph_d

# Open or create a database file
graph_d mydb.graphd

# Execute a single query and exit
graph_d mydb.graphd -c "MATCH (n:Person) RETURN n"

# Run queries from a script file
graph_d mydb.graphd -f queries.gql

Output Formats

# Table format (default) - human-readable
graph_d -c "MATCH (n) RETURN n LIMIT 5" -o table

# JSON format - for programmatic processing
graph_d -c "MATCH (n) RETURN n" -o json

# CSV format - for data export
graph_d -c "MATCH (n) RETURN n.name, n.age" -o csv

Shell Commands

When in interactive mode, these commands are available:

Command	Description
`.help`	Show help message
`.exit` or `.quit`	Exit the shell
`.mode`	Show available output modes
`.stats`	Show database statistics

Example Session

$ graph_d mydb.graphd
Graph_D 0.1.0 - Type .help for help, .exit to exit

graph_d> CREATE (n:Person {name: 'Alice', age: 30});
1 row returned

graph_d> CREATE (n:Person {name: 'Bob', age: 25});
1 row returned

graph_d> MATCH (n:Person) RETURN n.name, n.age;
name  | age
------+----
Alice | 30
Bob   | 25
2 rows returned

graph_d> .exit
Goodbye!

📊 Performance Benchmarks

Tested on modern hardware with realistic workloads:

Operation	Performance	Notes
Node Creation	140K-150K/sec	With JSON properties
Node Lookup	32M/sec	O(1) hash map access
Relationship Creation	100K/sec	With validation
Graph Traversal	~12-14μs	2-hop traversal
Memory Usage	~1.9KB/node	Including relationships

Scalability Results

100K nodes: Created in ~670ms
1M node target: Projected <7 seconds
Memory efficiency: <1GB for 1M documents target achieved

🔍 Advanced Queries

Aggregations

use graph_d::query::{QueryBuilder, AggregateFunction};

let all_employees: Vec<_> = (1..=100).collect();
let query = QueryBuilder::new(&graph, all_employees);

// Count employees
let count = query.aggregate(AggregateFunction::Count)?;

// Average salary
let avg_salary = query.aggregate(AggregateFunction::Avg("salary".to_string()))?;

// Group by department
let groups = query.aggregate(AggregateFunction::GroupBy("department".to_string()))?;

Sorting & Pagination

use graph_d::query::{SortCriteria, Pagination};

// Sort by multiple criteria
let sorted = QueryBuilder::new(&graph, all_nodes)
    .sort(vec![
        SortCriteria::asc("department"),
        SortCriteria::desc("salary"),
    ])?
    .nodes()?;

// Paginated results
let page = query.sorted_page(
    vec![SortCriteria::asc("name")],
    Pagination::new(0, 10)
)?;

Graph Traversal

use graph_d::query::QueryBuilder;

// Find friends of friends
let friends_of_friends = QueryBuilder::from_node(&graph, alice_id)
    .outgoing("FRIENDS_WITH")?
    .outgoing("FRIENDS_WITH")?
    .nodes()?;

// Filter by properties
let senior_engineers = QueryBuilder::new(&graph, all_employees)
    .filter_by_property("department", &json!("Engineering"))?
    .filter_by_property("level", &json!("Senior"))?
    .nodes()?;

🔒 Concurrent Transactions

use graph_d::transaction::{TransactionManager, IsolationLevel, LockableResource};

let tx_manager = TransactionManager::new(IsolationLevel::ReadCommitted);

// Create concurrent transaction
let mut tx = tx_manager.begin_concurrent();

// Acquire locks
tx.read_lock(LockableResource::Node(1))?;
tx.write_lock(LockableResource::Node(2))?;

// Perform operations...

// Commit (automatically releases locks)
tx.commit()?;

💾 Persistent Storage

// Create persistent database
let mut graph = Graph::open("my_graph.db")?;

// Use normally...
graph.create_node(properties)?;

// Flush to disk
graph.storage.flush()?;

// Data persists across restarts
let graph2 = Graph::open("my_graph.db")?;

🏗️ Architecture

Storage Layer

Memory-mapped files for efficient I/O
Fixed-size records for predictable performance
Index-free adjacency for O(1) traversals
String interning for memory deduplication

Query Engine

Fluent API for complex queries
Lazy evaluation where possible
Type-aware sorting for mixed data types
Statistical functions for analytics

Transaction System

MVCC with optimistic concurrency
Deadlock detection via wait-for graphs
Multiple isolation levels (Read Uncommitted → Serializable)
Automatic lock management

📚 Examples

The examples/ directory contains comprehensive demonstrations:

Rust Library Examples

getting_started.rs - Basic usage and quick start
persistent_storage.rs - File-based persistence
advanced_queries.rs - Complex query operations
concurrent_transactions.rs - Multi-threaded usage
gql_demo.rs - GQL query language examples
memory_management.rs - Memory allocation patterns
performance_test.rs - Benchmarking and profiling

CLI Examples

cli_scripting.gql - GQL script file for batch operations
cli_automation.sh - Shell script for CLI automation

Run Rust examples with:

cargo run --example getting_started
cargo run --example persistent_storage
cargo run --example concurrent_transactions

Run CLI examples with:

# Build CLI first
cargo build --features cli

# Run GQL script
./target/debug/graph_d mydb.graphd -f examples/cli_scripting.gql

# Run automation script
chmod +x examples/cli_automation.sh
./examples/cli_automation.sh

🧪 Testing

Run the comprehensive test suite:

# All tests
cargo test

# Benchmarks
cargo bench

# With output
cargo test -- --nocapture

🎯 Production Readiness

Current Status

✅ Core graph operations (CRUD)
✅ Advanced query engine with aggregations
✅ Concurrent transaction support with ACID compliance
✅ Multi-layered indexing system (Property, Range, Composite, Relationship)
✅ Memory-mapped persistence foundation
✅ GQL (Graph Query Language) parser, lexer, and executor
✅ Comprehensive test coverage (32+ tests)
✅ Performance benchmarks and optimization
✅ Complete rustdoc documentation with examples for all APIs

Roadmap

🔄 Performance Benchmarking Framework - Automated regression testing
📋 Schema Validation - Optional type constraints
📋 Enhanced Persistence - Complete crash recovery and integrity verification
📋 Clustering - Multi-node deployment support
📋 Backup/Restore - Data migration utilities

🤝 Contributing

Contributions are welcome! Please read our Contributing Guide and follow the Code of Conduct.

Development Setup

git clone https://github.com/your-org/graph_d
cd graph_d
cargo test
cargo run --example advanced_queries

📄 License

Licensed under either of:

Apache License, Version 2.0 (LICENSE-APACHE)
MIT License (LICENSE-MIT)

at your option.

🙏 Acknowledgments

Built with ❤️ using:

tokio - Async runtime
parking_lot - High-performance synchronization
serde_json - JSON serialization
memmap2 - Memory mapping
criterion - Benchmarking

Graph_D - Native graph database performance with Rust reliability. 🦀📊

graph_d 0.1.0