moltendb-core

🌋 The Pure Engine Crate

In-memory document store · Append-only WAL · Query evaluator · Analytics (🚧 WIP)
Zero knowledge of HTTP, auth, JWT, or WASM bindings.

What is this crate?

moltendb-core is the heart of MoltenDB. It contains every piece of logic that is shared between the HTTP server (moltendb-server) and the browser WASM adapter (moltendb-wasm):

• In-memory store — DashMap-backed document collections, keyed by (collection, key).
• Append-only WAL — every write is appended to a log file (LogEntry: INSERT, DELETE, DROP, INDEX, ENC). On startup the log is replayed into memory.
• Storage backends — DiskStorage (sync/async), TieredStorage (hot + cold log), EncryptedStorage (ChaCha20-Poly1305), OpfsStorage (WASM / browser OPFS).
• Query evaluator — $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $contains, $or, $and, field projection (include / exclude), dot-notation for nested fields, joins, sort, count, offset.
• Analytics engine — COUNT, SUM, AVG, MIN, MAX with optional WHERE filtering. ⚠️ Under active development — not ready for production use.
• Auto-indexing — query_heatmap tracks hot fields and builds indexes automatically.
• Handler pipeline — process_get, process_set, process_update, process_delete, process_analytics — the single source of truth consumed by both the server and the WASM adapter.
• Input validation — collection name, key, and field name rules enforced before any operation reaches the engine.

Crate type

[lib]

crate-type = ["rlib"]

moltendb-core compiles to a native rlib. It is not a cdylib — WASM bindings live in the separate moltendb-wasm crate. This keeps the native dependency tree clean (no wasm-bindgen, no web-sys).

WASM-specific code (OpfsStorage, Db::open_wasm) is gated behind #[cfg(target_arch = "wasm32")] and only compiled when the crate is used as a dependency of moltendb-wasm.

Add to your project

[dependencies]

moltendb-core = "0.3.0-beta.2"

Minimal example

use moltendb_core::engine::Db;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Open (or create) a database log file on disk
    let db = Db::open("./my_app.log").await?;

    // Insert a document
    db.set("users", "u1", serde_json::json!({
        "name": "Alice",
        "role": "admin"
    })).await?;

    // Read it back
    let user = db.get("users", "u1").await?;
    println!("{}", user);

    Ok(())
}

Using the handler pipeline (same API as the HTTP server)

use moltendb_core::{engine::Db, handlers};
use serde_json::json;

let db = Db::open("./my_app.log").await?;

let payload = json!({
    "collection": "users",
    "where": { "role": "admin" },
    "fields": ["name", "role"],
    "sort": [{ "field": "name", "order": "asc" }]
});

let (status_code, result) = handlers::process_get::process_get(&db, &payload, 10 * 1024 * 1024);
println!("{} — {}", status_code, result);

Module overview

Storage modes

Design constraints

• RAM is the hard limit. All documents are kept in memory for the lifetime of the process. There is no eviction, TTL, or page cache. A 100 000-document collection of typical JSON objects occupies roughly 100–200 MB of RAM.
• No HTTP, no auth, no JWT. This crate has zero knowledge of the network layer. It is safe to embed in any Rust application without pulling in Axum, Tokio TLS, or any auth dependency.
• Single writer, many readers. The DashMap store is safe for concurrent reads. Writes are serialised through the storage backend.

Part of the MoltenDB workspace

MoltenDB/
├── moltendb-core/     ← you are here
├── moltendb-wasm/     — browser adapter (wasm-bindgen glue, WorkerDb, OPFS)
├── moltendb-auth/     — identity layer (JWT, Argon2, UserStore)
└── moltendb-server/   — network layer (Axum, TLS, CORS, CLI config)

See the root README for the full architecture overview and feature list.