Expand description
The start-up disk test the appliance runs on /data (seq write/read, 4 KiB
fsync ops) — one disktest row on the screen.
One place that writes a small control file durably.
Both replication.json and webhooks.json are tiny JSON documents in the
data root that an operator sets over the control plane and that MUST survive
a power cut: a box that acknowledged the change and then lost power has to
come back with it. The write is the same on both, and getting it subtly
wrong is what left a 0-byte replication.json that panicked PID 1 on the
next boot. So there is one writer, and it is here (one implementation).
§Why the fsync dance
std::fs::write + rename is NOT durable on xfs: the rename’s directory
entry is journaled while the data blocks are not, so an interrupted write
leaves the file present but empty. So the bytes are fsync’d BEFORE the
rename, and the directory is fsync’d AFTER it — a crash then leaves either
the whole old file or the whole new one, never a torn tail.
Functions§
- backoff
- Exponential backoff:
base · 2^(attempt-1), capped atmax; zero forattempt == 0. - remove_
durable - Remove a per-box MARKER durably: unlink it, then fsync the directory, so
the removal survives a power cut and not only the next
stat. - sync_
dir - fsync a directory, so a rename into it or an unlink from it survives a power cut — its failure is the caller’s failure, never swallowed.
- write_
atomic - Atomically replace
<root>/<filename>withbytes, durably. - write_
atomic_ mirrored write_atomic, and then the SAME bytes again into the metadata mirror (crate::slow’sdir.meta.path— a second copy on the other spindle).- write_
atomic_ unframed write_atomicwithout the frame: the per-box metadata MIRROR (slow’smeta), which is a second copy on this box’s other spindle and never a transaction — the fallback slow root sits UNDER the data root, where a framed mirror write would put every document on the twin twice.