rightkit-db

Correct-by-default SQLite and SQLCipher connection bootstrap for Right Suite desktop apps. It owns connection pragmas, SQLCipher key ordering, a bounded read pool, checkpointing, optimization, optional integrity checks, online backup and restore, and versioned migrations. It does not own app schemas or queries.

Catalog capability: sqlite-storage. crates.io has 0.3.2; the source is 0.4.1 (see INDEX.md). The 0.4.0 APIs below (ReadPoolOptions, ReadPool::close_and_wait, VersionStorage::VersionTable, MigrateOptions::import_legacy_versions, the statement-cache feature) are not on crates.io yet, and open_writer_recovering_unreadable is unreleased (CHANGELOG Unreleased).

When to use it

• Use it to open a SQLite or SQLCipher database with the suite's journal, busy-timeout and durability rules.
• Use write_txn() for any transaction that may write. It starts BEGIN IMMEDIATE, so read-then-write work cannot fail with an unretryable stale snapshot. A caller that holds only &Connection (a closure-style helper) uses write_txn_unchecked(&conn), the same BEGIN IMMEDIATE through Transaction::new_unchecked; SQLite refuses a nested one.

Do not use it to define app tables or queries. Keep those in the app.

Default behaviour

The default writer and legacy reader open select WAL with rollback fallback, a bounded busy wait and journal, memory-backed temporary storage, and the selected durability profile.

• DurabilityProfile::Full: for user mutations that cannot be reconstructed.
• DurabilityProfile::Normal: for rebuildable indexes or telemetry.
• CacheProfile::LargeWorkingSet: opt-in. It sets cache_size to -65536 (KiB) (src/lib.rs:576).

Cargo features

No feature is on by default (0.4.0). Name bundled (otherwise the system SQLite is linked) and statement-cache to get what 0.3.x built.

SQLCipher consumers must turn off default features and choose exactly one provider feature. Both providers expose the same keyed-open and rekey APIs. Enabling both is a compile error. Keep the provider compatible with existing encrypted databases:

rightkit-db = { version = "<exact published version from INDEX.md>", default-features = false, features = ["sqlcipher-libtomcrypt"] }

rusqlite is a caret requirement (0.40.1), so consumers that resolve 0.40.2 stay compatible.

Usage

Taken from tests/contract.rs and the crate's read-only and migration API. Illustrative as a snippet:

use std::time::Duration;
use rightkit_db::{DbOptions, MigrateOptions, VersionStorage};

let mut db = DbOptions::new("file:context.db?mode=ro");
db.busy_timeout = Duration::from_millis(250);
let reader = db.open_read_only()?;

let migrations = MigrateOptions {
    version_storage: VersionStorage::Table {
        table: "meta".into(),
        key: "schema_version".into(),
    },
    ..Default::default()
};

Other APIs

• open_read_only(path) opens with SQLITE_OPEN_READ_ONLY. open_existing_writer(path) opens read-write without CREATE. Both accept paths or file: URIs with query parameters, and refuse missing files. The matching DbOptions methods use the path, key, timeout and optional integrity check. They skip journal selection, per-host routing, permission changes, init, optimization and migrations. Read-only main databases still allow SQLite TEMP tables unless callers set query_only.
• open_reader() is read-write or create bootstrap, then query_only as the last setup statement, read back and required to be on. It does not run PRAGMA optimize; the writer runs it when it opens.
• open_read_pool(cap) and open_read_pool_with(cap, ReadPoolOptions) return a ReadPool. Every reader has query_only set and read back as on, or its open fails. The default pool (ReadPoolOptions::new()) keeps read-write file handles. ReadPoolOptions::with_read_only(true) opens readers with SQLITE_OPEN_READ_ONLY, so writes fail even after query_only is turned off. A read-only pool needs an existing database: it never creates the file, changes the journal mode, narrows permissions, or runs optimize. Each reader runs the init hook, as on the other connections. A rollback journal left by a crash must be recovered by a writer first. ReadPoolOptions::with_reader_cache_kib(kib) sets each reader's cache_size to -kib (KiB, 1 to 2147483647, checked when the pool opens). It overrides CacheProfile::LargeWorkingSet on readers only. ReadPool::close() stops checkout and drops idle readers; a checked-out reader closes when it is returned. ReadPool::close_and_wait(timeout) closes, then waits until every reader is returned, so the database files are released (needed on Windows). If readers are still out at the timeout, it returns DbError::ReadPoolCloseTimeout { outstanding, timeout }. After close, get() returns DbError::ReadPoolClosed. ReadPool::get_timeout(duration) is get() with a bounded wait: Ok(Some(reader)), Ok(None) if no reader frees up in time (zero never waits), or DbError::ReadPoolClosed if the pool closes first.
• DbOptions::busy_timeout: Duration defaults to five seconds. It is installed before any bootstrap SQL, including key verification and init. It is the wait for a lock, not the time a backup copy may take: restore_database*, replace_database*, EncryptedStore::backup_to and restore, the pre-migration backup and relocation bound the whole copy by a budget derived from the source size (120 s plus 1 s per 4 MiB, capped at six hours, never less than busy_timeout) and use busy_timeout only for a busy or locked source. The public online_backup_to keeps busy_timeout as the deadline of the whole copy.
• DbOptions::optimize_on_open: bool defaults to true: a writer runs PRAGMA optimize when it opens. DbOptions::without_optimize_on_open() turns it off. Readers never run it.
• QuarantineStamp::Seconds names are unique per second: a second quarantine in the same second takes <db>.corrupt-<secs>-1, -2, and so on, claimed without overwriting an earlier file. QuarantineStamp::Custom names take the same -1, -2 retries. Millis names still refuse an existing destination.
• Unreadable (not corrupt) databases: open_writer_recovering_unreadable(policy, UnreadablePolicy). The default Fail returns DbError::Unreadable naming the cause and never moves the file; QuarantineAfter { attempts } renames it only after that many consecutive read failures (permission denied, I/O read error), counted in <db>.unreadable-opens. Disk full, read-only filesystem, device or share not ready, sharing or lock violations, busy/locked and a missing file never count.
• open_in_memory() returns a private ephemeral connection. The matching DbOptions::open_in_memory() keeps key, timeout, initializer, cache, foreign-key, durability, optimization and integrity options. Path, filesystem permissions, journal selection and mmap do not apply. The journal mode stays memory, including with WalStrict. Run migrate with db_path: None to skip filesystem backup and marker handling.
• MigrateOptions::version_storage defaults to VersionStorage::UserVersion. VersionStorage::Table { table, key } uses key and value columns compatible with Blueprint's meta table. VersionStorage::VersionTable { table, version_column, applied_at_column } is the other shape: one row per applied version (src/migrate.rs:67-84). Values are decimal text. Identifiers accept ASCII letters, digits and underscores. Keys are bound parameters. A missing table or key reads as zero. Table creation and version writes share each migration transaction. migration_version reads the chosen storage, and rollback_to_with_storage reverses it. user_version and rollback_to keep the default storage.
• DbOptions::restore_from(backup_path) checks every PRAGMA integrity_check row, including CHECK constraints, before it restores an existing live database through SQLite's backup API. A read transaction holds the validated source snapshot until the copy finishes. A missing or corrupt backup leaves the live database untouched. A missing live database is refused. The backup must use the options' SQLCipher key. The free function restore_from(options, destination, backup_path) restores into an existing mutable connection, including in-memory stores. The destination must be writable and outside any transaction. Busy and locked retries are bounded by the options' timeout.
• replace_database(options, replacement, lock_options) atomically replaces the live database file with a validated copy under StoreLock::exclusive (<db>.store-lock). Processes that keep connections open hold StoreLock::shared meanwhile; the lock is advisory and cooperative. Close every connection to the live database first (required on Windows). The replacement must use the options' SQLCipher key. replace_database_locked is the same step for a caller that already holds the exclusive guard.
• ReadPool::from_opener(cap, opener) builds a pool whose readers come from a closure, for tests over a connection the test builds. The pool enforces cap, get_timeout, and the close rules; query_only is the opener's job.
• open_writer_migrated_or_degrade(migrations, &MigrateOptions, RecoveryPolicy) is open_writer_migrated with recovery. A file that is corrupt, not a database, or fails its integrity check (at open, or as a migration step's SQLite error) is quarantined and recreated under the policy's stamp and sidecar options when quarantine_corrupt (plain) or quarantine_keyed (keyed) allows it, then migrated again. When the policy does not allow it, or the store still cannot be opened, the connection is a private in-memory store, migrated the same way. OpenOrDegrade::degraded (Degraded { reason, quarantined, quarantine_incomplete, in_memory }) reports either case. Any other error (busy, permission, I/O, a newer database, an invalid migration list) is returned, not degraded.
• MigrateOptions::backup_name: Option<BackupName> (Arc<dyn Fn(i64, i64) -> String + Send + Sync>) names the pre-migration backup (default <db>.pre-v<target>.bak; a name that is not a plain file name, or is the database, a live sidecar or a recovery file beside it, is refused before the backup is made). MigrateOptions::verify_backup: Option<VerifyBackup> (Arc<dyn Fn(&Path) -> Result<(), String> + Send + Sync>, so an owner can hold a key, a message count or a free-space floor) checks the backup before the first step; a failure stops the migration and keeps the backup. MigrateOptions::backup_retention: Option<usize> keeps only the newest n <db>.pre-*.bak backups after each backup (best effort; default keeps all).
• write_pending_marker(db, from, to) writes the <db>.migrating marker atomically with the pid and started_at_unix, the same file migrate writes. pending_marker_started_at(db) returns its start time (0 for a marker written before the time was recorded); PendingMarker keeps its three fields.
• stored_version_is_newer(conn, storage, migrations) reports whether the stored version is above the newest migration, reading only.
• checkpoint_restart(conn) is PRAGMA wal_checkpoint(RESTART). WalCheckpointJob (with WalCheckpointPolicy) is a caller-driven checkpoint job: note_activity on each use, and tick(conn) from the loop or timer the owner already runs. PASSIVE once the -wal reaches passive_above_bytes; RESTART once it reaches restart_above_bytes and the database has been idle for quiet_for. A tick is single-flight. The crate starts no thread.
• rightkit_db::Zeroizing re-exports zeroize::Zeroizing, the wrapper of the key fields and of the open_writer_with_key_candidates slice, so callers need no zeroize dependency.
• restore_database(options, replacement, &RestoreOptions, lock_options) is replace_database for a restore that must not lose the old store: the previous file and its sidecars are kept at <db>.pre-restore-<ms> (the path is in the returned RestoreReceipt), a failed placement or RestoreOptions::verify puts them back and keeps the bad copy at <db>.restore-failed-<ms>, and a crash mid-quarantine is finished by recover_interrupted_restore (call it before opening the store; a restore calls it first). restore_database_locked is the same under a held StoreLock::exclusive.

Platform support

• Unix: descriptor-relative file handling through rightkit-fs.
• Windows: the same database API. SQLCipher provider choice is the main per-OS difference (see Cargo features).
• Apple: sqlcipher-system uses CommonCrypto.

Verification

Tests: tests/contract.rs, tests/e2e.rs, tests/journal_fallback.rs, tests/membrane.rs, tests/read_pool.rs.

Version and changes

Published version: see INDEX.md. Release notes: CHANGELOG.md.

Publish only through the RightKit release command:

right-release publish cargo --crate rightkit-db