Struct LogPayloadStorage 

pub struct LogPayloadStorage { /* private fields */ }

Log-structured payload storage with snapshot support.

Stores payloads in an append-only log file with an in-memory index. Supports periodic snapshots for O(1) cold-start recovery instead of O(N) WAL replay.

Implementations

impl LogPayloadStorage

pub fn new<P: AsRef<Path>>(path: P) -> Result<Self>

Creates a new LogPayloadStorage with the default durability mode (Fsync).

If a snapshot file exists and is valid, loads from snapshot and replays only the WAL delta for fast startup. Otherwise, falls back to full WAL replay.

Errors

Returns an error if file operations fail.

pub fn new_with_durability<P: AsRef<Path>>( path: P, durability: DurabilityMode, ) -> Result<Self>

Creates a new LogPayloadStorage with the specified durability mode.

See DurabilityMode for available modes and their trade-offs.

Errors

Returns an error if file operations fail.

pub fn create_snapshot(&mut self) -> Result<()>

Creates a snapshot of the current index state.

The snapshot captures:

• Current WAL position
• All index entries (ID -> offset mappings)
• CRC32 checksum for integrity

Errors

Returns an error if file operations fail.

pub fn should_create_snapshot(&self) -> bool

Returns whether a new snapshot should be created.

Heuristic: Returns true if WAL has grown by more than the default threshold bytes since the last snapshot.

pub fn store_batch(&mut self, entries: &[(u64, &Value)]) -> Result<()>

Stores multiple payloads in a single batch operation.

Optimized for bulk imports: acquires WAL + index + offset locks once, writes all records sequentially, and performs a single durability sync at the end instead of per-point fsync.

Errors

Returns an error if serialization or WAL write fails. On partial failure, entries written before the error are durable (WAL is append-only).

pub fn store_batch_deferred(&mut self, entries: &[(u64, &Value)]) -> Result<()>

Stores multiple payloads without forcing an fsync at the end.

Identical to store_batch except the final sync_all() is replaced by a buffer-only flush(). WAL entries are written and the BufWriter is flushed to the OS kernel, but not fsynced to disk.

Use this for intermediate batches in a streaming bulk import, where only the final batch needs full durability. Call PayloadStorage::flush() after the last batch to force fsync.

Errors

Returns an error if serialization or WAL write fails.

Trait Implementations

impl PayloadStorage for LogPayloadStorage

fn store(&mut self, id: u64, payload: &Value) -> Result<()>

Stores a payload with the given ID.

fn retrieve(&self, id: u64) -> Result<Option<Value>>

Retrieves a payload by ID.

fn delete(&mut self, id: u64) -> Result<()>

Deletes a payload by ID.

fn flush(&mut self) -> Result<()>

Flushes pending writes to disk.

fn ids(&self) -> Vec<u64>

Returns all stored IDs.