Struct StorageManager 

pub struct StorageManager { /* private fields */ }

The storage manager — root of the storage engine.

Implementations

impl StorageManager

pub fn new(db_path: PathBuf, memory_manager: Arc<MemoryManager>) -> Self

pub fn set_spiller(&self, spiller: Option<Arc<Spiller>>)

Attach a spiller so node tables spill during bulk ingest once a NodeGroup’s buffer exceeds the memory threshold (P51.44). Applies to tables that already exist as well as future ones. A None clears the spiller.

pub fn spiller(&self) -> Option<Arc<Spiller>>

The currently attached spiller (if any).

pub fn set_group_commit(&mut self, config: Option<GroupCommitConfig>)

Enable (or disable) group commit at the WAL durability boundary (G6).

Installs a leader/follower coordinator over this manager’s WAL so concurrent commit_transaction step-1 flushes coalesce into a single fsync. None restores the legacy one-fsync-per-commit behavior. The override must be installed before concurrent commits begin (call it once at connection setup). Recovery format is unaffected.

pub fn group_commit(&self) -> Option<Arc<GroupCommit<Mutex<WAL>>>>

The currently attached group-commit coordinator, if any.

pub fn open(db_path: PathBuf, memory_manager: Arc<MemoryManager>) -> Self

Open (or create) a database at db_path, initializing all storage subsystems and replaying the WAL if necessary.

This is the primary entry point for storage initialization. After opening, call recover() to replay any uncommitted WAL records.

pub fn page_manager(&self) -> Option<&Arc<PageManager>>

Get a reference to the page manager, if available.

pub fn buffer_manager(&self) -> &Arc<Mutex<BufferManager>>

pub fn wal(&self) -> &Arc<Mutex<WAL>>

pub fn db_path(&self) -> &PathBuf

pub fn table_catalog(&self) -> Arc<TableCatalog>

Get a reference to the table catalog for reading/writing table data.

pub fn persist_all_tables(&self) -> Result<(), StorageError>

Flush all node + rel tables into their durable column mirrors.

Called after every write (commit or single-writer DML) and at checkpoint time so committed rows survive restarts (P45.4).

pub fn load_persisted_tables(&self) -> Result<usize, StorageError>

Load all persisted tables from their durable column mirrors.

Called during Database::new() AFTER tables are restored from the persisted catalog. Returns the number of tables that had persisted data.

pub fn drop_table_persistence(&self, table_id: u64)

Delete the durable column mirror for a dropped table.

pub fn log_column_write( &self, table_id: u64, col_id: u32, page_id: u64, data: &[u8], )

Log a column write to the WAL before applying it to the BufferManager.

pub fn create_node_table( &self, name: String, columns: Vec<ColumnDefinition>, ) -> NodeTable

Create a node table in the catalog and return its ID.

pub fn restore_node_table( &self, table_id: u64, name: String, columns: Vec<ColumnDefinition>, index_name: Option<&str>, ) -> NodeTable

Restore a node table at a specific table ID during recovery from a persisted catalog. Optionally recreates an ART primary-key index and registers its file with the BufferManager.

pub fn restore_rel_table( &self, table_id: u64, name: String, src_table_id: u64, dst_table_id: u64, columns: Vec<ColumnDefinition>, ) -> RelTable

Restore a rel table at a specific table ID during recovery from a persisted catalog.

pub fn create_vector_index( &self, name: String, table_name: String, column_name: String, metric: DistanceMetric, dimensions: u32, ) -> VectorIndexTable

Create a vector index in the catalog and register its file with the BufferManager.

pub fn get_vector_index_by_name( &self, name: &str, ) -> Option<Ref<'_, u64, VectorIndexTable>>

Get a vector index by name.

pub fn get_vector_index_by_name_mut( &self, name: &str, ) -> Option<RefMut<'_, u64, VectorIndexTable>>

Get a mutable vector index by name.

pub fn create_art_index( &self, table_name: &str, index_name: &str, ) -> Result<(), StorageError>

Create an ART (Adaptive Radix Tree) index on a node table. Delegates to TableCatalog and registers the index file with BufferManager.

pub fn drop_art_index( &self, table_name: &str, _index_name: &str, ) -> Result<(), StorageError>

Drop an ART index from a node table.

pub fn get_art_index(&self, table_name: &str) -> Option<ArtPrimaryKeyIndex>

Get the ART index for a node table (cloned copy for read-only access).

pub fn create_rel_table( &self, name: String, src_table_id: u64, dst_table_id: u64, columns: Vec<ColumnDefinition>, ) -> RelTable

Create a rel table in the catalog.

pub fn wal_size(&self) -> usize

Get the total size of the WAL in bytes.

pub fn checkpoint(&self) -> Result<CheckpointResult>

Perform a checkpoint: flush WAL + dirty pages to disk.

pub fn maybe_checkpoint( &self, threshold: i64, drain_fn: Option<&dyn Fn(Duration) -> bool>, ) -> Result<bool>

Conditionally trigger a checkpoint based on the given threshold.

This is called after every DML/DDL operation from Connection::query().

Semantics:

• threshold < 0 (e.g., -1): checkpoint after every write (every DML/DDL).
• threshold == 0: never auto-checkpoint (manual only via CHECKPOINT).
• threshold > 0: checkpoint when wal_size() > threshold (bytes).

Returns true if a checkpoint was triggered.

pub fn checkpoint_with_drain( &self, drain_fn: Option<&dyn Fn(Duration) -> bool>, ) -> Result<CheckpointResult>

Perform a checkpoint with transaction drain.

Two-phase drain:

1. Call the drain_fn callback to stop new transactions and wait for active ones
2. Perform the checkpoint (WAL flush + BM flush)

This is the concurrent-writer-safe checkpoint. Use this instead of plain checkpoint() when concurrent writes are enabled.

If drain_fn is None, the drain is skipped (backwards-compatible default). If the drain times out, the checkpoint proceeds anyway — this is safe because the WAL will capture any in-flight writes.

pub fn storage_info(&self) -> StorageInfo

Get storage-level information for diagnostics.

pub fn buffer_info(&self) -> BufferInfo

Buffer manager statistics for CALL bm_info().

pub fn file_info(&self) -> FileInfo

File-level statistics for CALL file_info() / CALL disk_size_info().

pub fn fsm_info(&self) -> FsmInfo

FSM statistics for CALL free_space_info().

pub fn commit_transaction( &self, local_storage: &LocalStorage, shadow_file: &ShadowFile, checkpoint_threshold: i64, txn_id: u64, drain_fn: Option<&dyn Fn(Duration) -> bool>, ) -> Result<(), StorageError>

Commit a write transaction’s data to storage.

Orchestrates the full commit pipeline:

1. Append Commit record to the WAL (write-ahead log) + fsync
2. Flush LocalStorage buffered writes to the actual tables
3. Apply ShadowFile copy-on-write pages to the BufferManager
4. Optionally checkpoint if the WAL threshold is met

Since P60.2 the SQL write path emits typed Insert/Delete/Update WAL records, so committed data is durable from the WAL alone; the durable column mirrors are written only by checkpoints and by recover().

Arguments

• local_storage — the transaction’s write buffer (consumed on success).
• shadow_file — the transaction’s COW page buffer.
• checkpoint_threshold — passed to maybe_checkpoint(); use -1 for always-checkpoint, 0 for never, N for byte-based threshold.
• drain_fn — optional callback to drain active transactions before checkpoint.

Returns Ok(()) if the commit pipeline succeeded.

pub fn rollback_transaction( &self, local_storage: &mut LocalStorage, shadow_file: &mut ShadowFile, txn_id: u64, undo_records: &[UndoRecord], ) -> Result<(), StorageError>

Roll back a write transaction, discarding all pending changes.

Clears the local storage buffer, discards shadow pages, and applies undo records to restore pre-write state. The caller should also call TransactionManager::rollback() to update the transaction’s status and release locks.

undo_records — accumulated undo records from the transaction. Applied in reverse order to restore overwritten data.

Returns Ok(()) on success.

pub fn recover(&self) -> Result<usize>

Recover state after a crash or unclean shutdown.

Recovery source order (P45.4, amended P60.2):

1. Durable column mirrors — the state at the last checkpoint — are loaded first;
2. WAL replay then applies every committed delta since that checkpoint: typed Insert/Delete/Update records emitted by the SQL write path, plus records decoded out of bulk-copied LocalWALData blobs. Replaying on top of the mirrors preserves row-id continuity and reconstructs full state even when no checkpoint ever ran (mirrors absent → whole log is replayed).

Finally the recovered tables are re-persisted and a checkpoint resets the WAL, so a subsequent startup restores from mirrors alone.

Call this once during Database::new(), after table schemas have been re-created from the persisted catalog (same table IDs).

Returns the number of data records applied during replay (0 when the WAL was empty), or an error if recovery fails (database is corrupt).