Module commit 

Commit serialization for metadata read-modify-write cycles and dataset writes.

Base concurrency-safety model borrowed from duva’s actor design ([<https://github.com/Migorithm/duva>]): every storage instance has a single logical commit actor. All metadata mutations (the load_metadata → mutate → save_metadata cycle performed by every save_* call) are serialized through it, so concurrent writers cannot interleave their cycles and lose each other’s registry entries (lost update) — the same reason duva routes writes through one actor mailbox instead of shared mutable state.

The same mailbox shape extends to Lance dataset writes ([with_dataset_write_lock]): manifest-version allocation and the commit-point publish of one dataset directory are serialized, so two concurrent overwrites cannot mint the same N.manifest (#95).

Both registries hold weak references (#98): a mailbox stays alive only while some caller holds its Arc (i.e. while a commit cycle or dataset write is in flight), and dead entries are swept on insert. Instances that churn (create/drop thousands of collections) keep the registries bounded.

Durability of the commit itself is the tmp + fsync + rename discipline (crate::generations::write_json_atomic for metadata, the same sequence inside the lancefmt writer for data/txn/manifest files): readers never observe a half-written commit pointer, and a read after a completed commit observes its effects (read-your-own-writes).

Downstream contract (#100)

The commit actor is an in-process mailbox: it serializes concurrent tasks inside one process, never two independent processes. Two levels of arbitration are exposed:

1. In-process — with_commit_actor wraps a full metadata read-modify-write cycle (load → mutate → publish). Every cycle over a given metadata file runs under the same per-path mailbox, so cycles from your code and cycles from the save_* registry paths cannot interleave and lose updates:

   use std::path::Path;
   use genegraph_storage::commit::with_commit_actor;
   
   futures::executor::block_on(async {
       let metadata_path = Path::new("base/ds__g1_metadata.json");
       let cycle: genegraph_storage::StorageResult<()> = with_commit_actor(metadata_path, || async {
           // load → mutate → publish (via generations::write_json_atomic)
           Ok(())
       })
       .await;
       let _ = cycle;
   });

2. Cross-process — an advisory flock on a lock file, held for the documented hold scope (the whole read-modify-write cycle: lock → load → mutate → publish → release). The blessed convention is one lock file next to the metadata file, named by lock_file_for_metadata ({metadata-stem}.lock). Two forms exist:

   • with_metadata_file_lock — the composed recipe: the file lock is held across the whole awaited commit-actor cycle, so cross-process exclusion and in-process actor serialization are both active for the duration of the cycle. This is what multi-process consumers whose metadata file is also touched by async save_* paths must use:

     use std::path::Path;
     use genegraph_storage::commit::with_metadata_file_lock;
     
     futures::executor::block_on(async {
         let metadata_path = Path::new("base/ds__g1_metadata.json");
         let cycle = with_metadata_file_lock(metadata_path, || async {
             // load → mutate → publish (via generations::write_json_atomic)
             Ok(())
         })
         .await;
         let _ = cycle;
     });

   • with_file_lock — the raw lock for consumers whose whole cycle is synchronous. Its closure runs on the blocking pool and cannot await the commit actor; it therefore does not serialize against in-process async save_* cycles. If that matters, use with_metadata_file_lock.

   • Fail-fast variants (#105) — try_with_file_lock and try_with_metadata_file_lock take the same lock files with the same hold scopes, but acquisition is non-blocking (flock(LOCK_EX | LOCK_NB)): on contention the caller gets StorageError::LockWouldBlock naming the lock file immediately instead of parking on the blocking pool. Consumers whose contract is fail-fast on contention (a second concurrent append must exit non-zero, never wait) map that variant onto their own taxonomy.

The lock file is a rendezvous point for cooperating writers, not a commit artifact: it carries no data and is left in place after release. Arbitration is only as strong as the convention — every writer of the same metadata file must take the same lock file before mutating it, and advisory locking excludes only those cooperating writers, never arbitrary readers or unaware processes.

The blocking lock forms wait on the blocking pool; see the operational caution on with_metadata_file_lock for the assumptions this puts on commit cycles and what to prefer when waits could be prolonged or numerous. Where that trade is unacceptable, the try forms above fail fast instead of waiting.

Functions

lock_file_for_metadata

The blessed lock-file path for a metadata file (#100): {metadata-stem}.lock next to it — ds__g1_metadata.json locks through ds__g1_metadata.lock. All cooperating writers of the same metadata file must resolve the lock through this function so the convention holds.

try_with_file_lock

Fail-fast counterpart of with_file_lock (#105): the same advisory flock on lock_path and the same hold scope — lock → read-modify-write → publish → release — but acquisition is non-blocking (flock(LOCK_EX | LOCK_NB)). On contention the call returns immediately with StorageError::LockWouldBlock naming the lock file instead of parking the waiter on the blocking pool, so consumers whose contract is fail-fast on contention (e.g. a multi-process CLI append that must exit non-zero on a concurrent append) can adopt the blessed convention without waiting.

try_with_metadata_file_lock

Fail-fast counterpart of with_metadata_file_lock (#105): the lock file is resolved through lock_file_for_metadata and acquired with [FileLock::try_acquire] — on cross-process contention the call returns StorageError::LockWouldBlock naming the derived lock file without parking; uncontended, the file lock is held across the whole awaited commit-actor cycle exactly as in with_metadata_file_lock (cross-process exclusion and in-process actor serialization).

with_commit_actor

Runs commit (a full metadata read-modify-write cycle) under the metadata path’s commit actor: at most one cycle runs at a time for the same path within this process.

with_file_lock

Cross-process commit arbitration (#100): an advisory flock on lock_path, held for the closure’s scope — lock → read-modify-write → publish → release. Independent processes (e.g. separate CLI invocations) that take the same lock file cannot interleave their metadata cycles.

with_metadata_file_lock

The blessed recipe for a multi-process consumer’s metadata read-modify-write cycle (#100, review composition fix): the advisory file lock (lock_file_for_metadata) is held across the whole awaited commit-actor cycle — cross-process exclusion and in-process actor serialization are both active for the duration of cycle.