citadeldb 0.6.0

Citadel: encrypted-first embedded database engine that outperforms unencrypted SQLite
Documentation

Every page is encrypted and authenticated before it hits disk. The database file is always opaque. Beats unencrypted SQLite in all 23 benchmarks with equal cache budgets.

Features

  • Encrypted at rest - AES-256-CTR + HMAC-SHA256 per page, verified before decryption
  • SQL - JOINs, subqueries, CTEs (recursive), UNION/INTERSECT/EXCEPT, window functions, views, aggregates, indexes, constraints, ALTER TABLE, prepared statements
  • ACID - Copy-on-Write B+ tree, shadow paging, no WAL. Snapshot isolation with concurrent readers
  • P2P sync - Merkle-based table diffing over Noise-encrypted channels with PSK auth
  • CLI - SQL shell with tab completion, syntax highlighting, dot-commands (.backup, .verify, .rekey, .sync, .dump, ...)
  • 3-tier key hierarchy - Passphrase -> Argon2id -> Master Key -> AES-KW -> REK -> HKDF -> DEK + MAC
  • FIPS 140-3 - PBKDF2-HMAC-SHA256 + AES-256-CTR when compliance requires it
  • Audit log - HMAC-SHA256 chained, tamper-evident
  • Hot backup - Consistent snapshots via MVCC, no write blocking
  • Overflow pages - Large values handled transparently, no size limits
  • Cross-platform - Windows, Linux, macOS. C FFI (37 functions), WebAssembly bindings
  • 2,560+ tests - Unit, integration, torture tests across 10 crates

Benchmarks

Citadel vs SQLite on 100K rows (INTEGER, TEXT, INTEGER), single-threaded:

Benchmark          Citadel        SQLite         Ratio
-----------------------------------------------------
correlated_in      5.49 ms        1.90 s         346x
count              146 ns         31.0 us        213x
correlated_scalar  264 us         18.1 ms        69x
point              915 ns         12.7 us        13.9x
group_by           1.94 ms        10.2 ms        5.3x
cte                1.43 ms        6.16 ms        4.3x
view_point         3.02 us        12.7 us        4.2x
view_filter        734 us         1.77 ms        2.4x
filter             721 us         1.76 ms        2.4x
window_agg         36.5 ms        84.0 ms        2.3x
sort               1.15 ms        2.52 ms        2.2x
window_rank        65.5 ms        133 ms         2.0x
insert             64.9 us        118 us         1.8x
scan               9.15 ms        13.5 ms        1.5x
insert_select      738 us         1.13 ms        1.5x
sum                1.35 ms        1.84 ms        1.4x
delete             102 us         137 us         1.3x
join               87.0 us        113 us         1.3x
distinct           3.14 ms        3.88 ms        1.2x
correlated_exists  5.78 ms        6.59 ms        1.1x
update             27.9 us        29.7 us        1.07x
union              176 us         181 us         1.03x
recursive_cte      111 us         115 us         1.03x
  • count - SELECT COUNT(*) FROM t
  • point - SELECT * FROM t WHERE id = 50000
  • scan - SELECT * FROM t (full 100K rows)
  • filter - SELECT * FROM t WHERE age = 42
  • sort - SELECT * FROM t ORDER BY age LIMIT 10
  • sum - SELECT SUM(age) FROM t
  • group_by - SELECT age, COUNT(*) FROM t GROUP BY age
  • distinct - SELECT DISTINCT age FROM t
  • join - SELECT a.id, b.data FROM a INNER JOIN b ON a.id = b.a_id (1K x 1K)
  • union - SELECT id, val FROM a UNION ALL SELECT id, data FROM b
  • cte - WITH filtered AS (SELECT ... WHERE age < 50) SELECT age, COUNT(*) FROM filtered GROUP BY age
  • recursive_cte - WITH RECURSIVE seq(x) AS (SELECT 1 UNION ALL SELECT x+1 FROM seq WHERE x < 1000) SELECT SUM(x) FROM seq
  • insert - 100 rows in one transaction
  • update - UPDATE t SET age = age + 1 WHERE id BETWEEN 10000 AND 10099
  • delete - insert 100 rows then delete them
  • insert_select - INSERT INTO sink SELECT id, val FROM a
  • window_rank - ROW_NUMBER() OVER (PARTITION BY age ORDER BY id), RANK() OVER (...)
  • window_agg - SUM(age) OVER (ORDER BY id ROWS 50 PRECEDING), MIN(age) OVER (...)
  • view_filter - SELECT * FROM v WHERE age = 42 (v = view over t)
  • view_point - SELECT * FROM v WHERE id = 50000
  • correlated_exists - SELECT COUNT(*) FROM t WHERE EXISTS (SELECT 1 FROM ref_table WHERE ref_table.id = t.id)
  • correlated_in - SELECT COUNT(*) FROM t WHERE id IN (SELECT id FROM ref_table WHERE ref_table.val = t.age)
  • correlated_scalar - SELECT a.id, (SELECT COUNT(*) FROM b WHERE b.a_id = a.id) FROM a

SQLite config: journal_mode=OFF, synchronous=OFF, cache_size=8000 (~32 MB). Citadel config: SyncMode::Off, cache_size=4096 (~32 MB). Both run with durability disabled to measure pure engine overhead, not disk I/O.

Reproduce with cargo bench -p citadeldb-sql --bench h2h_bench

Quick Start

Library

use citadel::DatabaseBuilder;
use citadel_sql::Connection;

let db = DatabaseBuilder::new("my.db")
    .passphrase(b"secret")
    .create()?;

let mut conn = Connection::open(&db)?;
conn.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL);")?;
conn.execute("INSERT INTO users (id, name) VALUES (1, 'Alice');")?;
let result = conn.query("SELECT * FROM users;")?;

// Key-value API
let mut wtx = db.begin_write()?;
wtx.insert(b"key", b"value")?;
wtx.commit()?;

let mut rtx = db.begin_read();
assert_eq!(rtx.get(b"key")?.unwrap(), b"value");

// Named tables
let mut wtx = db.begin_write()?;
wtx.create_table(b"sessions")?;
wtx.table_insert(b"sessions", b"token-abc", b"user-42")?;
wtx.commit()?;

// In-memory (no file I/O - useful for testing and WASM)
let mem_db = DatabaseBuilder::new("")
    .passphrase(b"secret")
    .create_in_memory()?;

CLI

citadel --create my.db


citadel> CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL);
citadel> INSERT INTO users (id, name) VALUES (1, 'Alice'), (2, 'Bob');
citadel> SELECT * FROM users;
+----+-------+

| id | name  |
+----+-------+

|  1 | Alice |
|  2 | Bob   |
+----+-------+


citadel> .backup mydb.bak

citadel> .verify

citadel> .stats

citadel> .audit verify

citadel> .rekey

citadel> .compact clean.db

citadel> .dump users


# P2P sync

citadel> .keygen

citadel> .listen 4248 <KEY>              # Terminal A

citadel> .sync 127.0.0.1:4248 <KEY>      # Terminal B

SQL

Statements - CREATE/DROP TABLE, ALTER TABLE (ADD/DROP/RENAME COLUMN, RENAME TABLE), CREATE/DROP INDEX, CREATE/DROP VIEW, INSERT (VALUES, SELECT), SELECT, UPDATE, DELETE, BEGIN/COMMIT/ROLLBACK, EXPLAIN

Constraints - PRIMARY KEY, NOT NULL, UNIQUE, DEFAULT, CHECK (column + table level), FOREIGN KEY (RESTRICT/NO ACTION)

Types - INTEGER, REAL, TEXT, BLOB, BOOLEAN

Clauses - JOINs (INNER, LEFT, RIGHT, CROSS), subqueries (scalar, IN, EXISTS, correlated), CTEs (WITH / WITH RECURSIVE), UNION/INTERSECT/EXCEPT [ALL], CASE, BETWEEN, LIKE, DISTINCT, GROUP BY/HAVING, ORDER BY, LIMIT/OFFSET

Window functions - ROW_NUMBER, RANK, DENSE_RANK, NTILE, LAG, LEAD, FIRST_VALUE, LAST_VALUE, SUM/COUNT/AVG/MIN/MAX OVER with PARTITION BY, ORDER BY, ROWS/RANGE frames

Views - CREATE/DROP VIEW, OR REPLACE, IF NOT EXISTS/IF EXISTS, column aliases, nested views

Functions - COUNT, SUM, AVG, MIN, MAX, LENGTH, UPPER, LOWER, SUBSTR, ABS, ROUND, COALESCE, NULLIF, CAST, TYPEOF, HEX, TRIM, REPLACE, RANDOM, IIF, GLOB

Prepared statements - $1, $2, ... positional parameters with LRU statement cache

Security

No plaintext on disk. Every page is encrypted before writing and authenticated before reading.

Separate key file. Encryption keys live in {dbname}.citadel-keys, not inside the database. The passphrase derives a master key in memory via Argon2id (or PBKDF2 in FIPS mode) and never touches disk.

Key backup. Export an encrypted key backup with a separate recovery passphrase. Restore access without re-encrypting the entire database.

Instant rekey. Changing the passphrase re-wraps the root encryption key. No page re-encryption - instant regardless of database size.

Encrypted sync. Noise protocol (NNpsk0_25519_ChaChaPoly_BLAKE2s) with a 256-bit pre-shared key. Ephemeral Curve25519 keys per session for forward secrecy.

Architecture

+-------------+---------------+---------------+
| citadel-cli | citadel-ffi   | citadel-wasm  |  CLI, C FFI, WebAssembly
+-------------+---------------+---------------+
|                 citadel-sql                  |  SQL parser, planner, executor
+---------------------------------------------+
|                  citadel                     |  Database API, builder, sync
+--------------+--------------+---------------+
|  citadel-txn | citadel-sync | citadel-crypto|  Transactions, replication, keys
+--------------+--------------+---------------+
|citadel-buffer|         citadel-page         |  Buffer pool (SIEVE), page codec
+--------------+------------------------------+
|              citadel-io                      |  File I/O, fsync, io_uring
+---------------------------------------------+
|              citadel-core                    |  Types, errors, constants
+---------------------------------------------+

Page Layout (8,208 bytes)

+----------+--------------------+----------+
|  IV 16B  |  Ciphertext 8160B  |  MAC 32B |
+----------+--------------------+----------+

Fresh random IV per page. HMAC verified before decryption.

Commit Protocol

Shadow paging with a god byte - one byte selects the active commit slot. Atomic commits without WAL:

  1. Write dirty pages to new locations (CoW)
  2. Compute Merkle hashes bottom-up
  3. Update the inactive commit slot
  4. Flip the god byte

Language Bindings

C / C++

Static or dynamic library with auto-generated citadel.h (cbindgen). All 37 functions are panic-safe.

#include "citadel.h"

CitadelDb *db = NULL;
citadel_create("my.db", "secret", 6, &db);

CitadelWriteTxn *wtx = NULL;
citadel_write_begin(db, &wtx);
citadel_write_put(wtx, (const uint8_t*)"key", 3, (const uint8_t*)"val", 3);
citadel_write_commit(wtx);

CitadelSqlConn *conn = NULL;
citadel_sql_open(db, &conn);
CitadelSqlResult *result = NULL;
citadel_sql_execute(conn, "SELECT * FROM users;", &result);

citadel_close(db);

WebAssembly

import { CitadelDb } from "@citadeldb/wasm";

const db = new CitadelDb("secret");
db.execute("CREATE TABLE t (id INTEGER PRIMARY KEY, name TEXT);");
db.execute("INSERT INTO t (id, name) VALUES (1, 'Alice');");

const result = db.query("SELECT * FROM t;");
// { columns: ["id", "name"], rows: [[1, "Alice"]] }

db.put(new Uint8Array([1, 2, 3]), new Uint8Array([4, 5, 6]));

Build: wasm-pack build crates/citadel-wasm --target web

Building

Rust 1.75+.

git clone https://github.com/yp3y5akh0v/citadel.git

cd citadel

cargo build --release

Feature Flags

Flag Description
audit-log HMAC-chained tamper-evident audit log (default: on)
fips FIPS 140-3: PBKDF2 + AES-256-CTR only
io-uring Linux io_uring async I/O

License

MIT OR Apache-2.0