Struct StorageEngine 

pub struct StorageEngine { /* private fields */ }

The unified storage engine for MenteDB.

Coordinates page allocation, caching, and write-ahead logging to provide crash-safe, page-oriented storage for memory nodes.

Concurrency model (inspired by WAL-mode databases):

• Reads are lock-free: read_page only touches the buffer pool and page manager — no file locks, no WAL access.
• Writes are fully serialized via a blocking flock(2) on the WAL file. The entire write transaction (page allocation + WAL append + page write + fsync) executes under a single flock, ensuring correctness across multiple processes sharing the same data directory.
• State is refreshed from disk under the flock: page count is re-read from the file header and LSN is re-read from the WAL tail, so no process can act on stale in-memory state.
• No DB-level lock on open. Multiple processes can open the same database simultaneously.

Implementations

impl StorageEngine

pub fn open(path: &Path) -> MenteResult<Self>

Open (or create) a storage engine rooted at path.

path must be a directory; it will be created if it does not exist. After opening, any uncommitted WAL entries are replayed for crash recovery.

Example

use mentedb_storage::StorageEngine;

let engine = StorageEngine::open("/tmp/mentedb".as_ref())?;
// engine is ready — WAL recovery already ran if needed

pub fn recover(&self) -> MenteResult<usize>

Replay WAL entries to recover writes that were not checkpointed.

For each PageWrite entry the serialized data is written back to its page. After replay the WAL is truncated. Returns the number of entries replayed.

pub fn close(&self) -> MenteResult<()>

Gracefully shut down: flush dirty pages, sync files.

Example

engine.close()?;

pub fn allocate_page(&self) -> MenteResult<PageId>

Allocate a fresh page (for internal/test use).

WARNING: In multi-process scenarios, prefer store_memory which allocates under the WAL flock. This method does NOT acquire the flock.

pub fn read_page(&self, page_id: PageId) -> MenteResult<Box<Page>>

Read a page through the buffer pool (lock-free — no WAL access).

pub fn write_page(&self, page_id: PageId, data: &[u8]) -> MenteResult<()>

Write data into an already-allocated page with WAL protection.

Acquires the WAL flock for the duration of the write transaction. For new pages, prefer store_memory which allocates + writes atomically.

pub fn store_memory(&self, node: &MemoryNode) -> MenteResult<PageId>

Serialize and store a MemoryNode into a single page.

The entire operation — page allocation, WAL append, page write — executes under a single WAL flock, making it safe across multiple processes.

Example

use mentedb_storage::StorageEngine;
use mentedb_core::{MemoryNode, memory::MemoryType, types::AgentId};

let engine = StorageEngine::open("/tmp/mentedb".as_ref())?;
let node = MemoryNode::new(
    AgentId::new(),
    MemoryType::Semantic,
    "User likes dark mode".to_string(),
    vec![0.1, 0.2],
);
let page_id = engine.store_memory(&node)?;

pub fn load_memory(&self, page_id: PageId) -> MenteResult<MemoryNode>

Load and deserialize a MemoryNode from the given page.

Example

let node = engine.load_memory(PageId(1))?;
println!("memory: {}", node.content);

pub fn checkpoint(&self) -> MenteResult<()>

Checkpoint: flush all dirty pages, sync to disk, and truncate the WAL.

Example

// After a batch of writes, checkpoint to reclaim WAL space
engine.checkpoint()?;

pub fn scan_all_memories(&self) -> Vec<(MemoryId, PageId)>

Scan all pages and return (MemoryId, PageId) pairs for every valid memory node.

Refreshes the page count from disk before scanning so pages written by other processes are included. Used to rebuild the page map on startup.

Example

let memories = engine.scan_all_memories();
for (memory_id, page_id) in &memories {
    println!("{memory_id} -> page {}", page_id.0);
}