Expand description
Commit serialization for metadata read-modify-write cycles and dataset writes.
Base concurrency-safety model borrowed from duva’s actor design
([<https://github.com/Migorithm/duva>]): every storage instance has a single
logical commit actor. All metadata mutations (the load_metadata → mutate → save_metadata cycle performed by every save_* call) are
serialized through it, so concurrent writers cannot interleave their
cycles and lose each other’s registry entries (lost update) — the same
reason duva routes writes through one actor mailbox instead of shared
mutable state.
The same mailbox shape extends to Lance dataset writes
([with_dataset_write_lock]): manifest-version allocation and the
commit-point publish of one dataset directory are serialized, so two
concurrent overwrites cannot mint the same N.manifest (#95).
Both registries hold weak references (#98): a mailbox stays alive
only while some caller holds its Arc (i.e. while a commit cycle or
dataset write is in flight), and dead entries are swept on insert.
Instances that churn (create/drop thousands of collections) keep the
registries bounded.
Durability of the commit itself is the tmp + fsync + rename discipline
(crate::generations::write_json_atomic for metadata, the same
sequence inside the lancefmt writer for data/txn/manifest files):
readers never observe a half-written commit pointer, and a read after a
completed commit observes its effects (read-your-own-writes).
Cross-process arbitration stays with the transactional-generations work
(#93/#81-P5).