Crate durability 

durability: crash-consistent persistence primitives for segment-based indices.

Scope:

• directory abstraction (Directory)
• on-disk framing constants (formats)
• write-ahead log (walog)
• checkpoints (checkpointing)
• crash recovery from WAL (recover)

Non-goal: indexing algorithms or ranking (those belong in crates like postings / jin).

Contract (what you can rely on)

This crate is designed around two different “strength levels”:

• Crash-consistent + integrity-checked (default)
  • Detects corruption (CRC/magic/version/type mismatches) and errors loudly.
  • Supports best-effort recovery of a torn tail (partial record write) in the final log segment.
  • Guarantees a prefix property under best-effort replay: recovered operations are a prefix of the successfully written operation stream (no garbage / no reordering).
• Stable-storage durability (opt-in)
  • Requires explicit barriers (fsync/sync_all) and sometimes parent directory sync.
  • Use storage::sync_file / storage::sync_parent_dir and flush_and_sync() helpers where you need “survives power loss after success” semantics.

Terminology:

• flush() is a visibility boundary, not a stable-storage guarantee.
• “Best-effort” is intentionally narrow; it never masks corruption.

Note: this crate intentionally exposes traits and framing. Higher-level crates generally decide directory layout, naming, and lifecycle policies, but some primitives (notably walog) assume a conventional wal/ directory.

Re-exports

pub use error::PersistenceError;

pub use error::PersistenceResult;

pub use publish::CheckpointPublisher;

pub use publish::PublishResult;

pub use storage::Directory;

pub use storage::DurableDirectory;

pub use storage::FsDirectory;

pub use storage::MemoryDirectory;

Modules

checkpoint

Generic checkpoint file (single snapshot blob).

checkpointing

Checkpoint files for durable state snapshots.

error

Error types for durability.

formats

On-disk format constants and small shared structs.

publish

Crash-safe checkpoint publishing and WAL truncation helpers.

recordlog

Append-only record log (generic WAL primitive).

recover

Crash recovery using checkpoint + WAL replay.

replay

Generic replay helpers.

storage

Storage abstraction for durability.

walog

Write-ahead log (WAL) for incremental updates.