Struct Log 

pub struct Log { /* private fields */ }

The transaction log: one writer, any number of readers, one directory.

append is O(1) in the size of the log: a positioned write at the known end of the open segment, one fsync, one 8-byte index append.

Implementations

impl Log

pub fn open(dir: impl AsRef<Path>) -> Result<Self>

Open or create the log at dir, cutting a torn tail if there is one.

pub fn open_with(dir: impl AsRef<Path>, segment_bytes: u64) -> Result<Self>

Log::open with a segment size (tests roll segments small).

pub fn mutation(&self) -> MutationGuard

Enter a framed mutation. Hold the guard from before the frame is appended until the store has the write. Waits while another thread holds Log::freeze; the freezing thread itself passes (a mutation it causes lands after the snapshot’s sequence and is replayed, which every apply tolerates). Nested guards on one thread count once.

pub fn freeze(&self) -> FreezeGuard

Freeze: no new framed mutation starts on another thread, and this returns once every one in flight has reached its store. Reads are untouched. One freezer at a time. Must not be called from inside a mutation (it would wait for itself).

pub fn dir(&self) -> &Path

The directory.

pub fn set_follower(&self, on: bool)

Make this log follower-only (a twin; a box being restored): append and begin(None) refuse with Error::Follower, append_at works.

pub fn is_follower(&self) -> bool

See Log::set_follower.

pub fn group_hold(&self) -> Option<usize>

The test hold of a group commit this log honours (TEST_GROUP_ENV); None on every box.

pub fn head_seq(&self) -> u64

The newest committed sequence; 0 when nothing has been committed.

pub fn first_seq(&self) -> u64

The oldest sequence still retained. head + 1 when the log is empty.

pub fn append(&self, kind: Kind, meta: &Meta, blob: &[u8]) -> Result<u64>

Append one transaction with the next sequence number. Durable when this returns.

pub fn append_at( &self, seq: u64, kind: u16, meta: &[u8], blob: &[u8], ) -> Result<u64>

Log::append, but the caller says which sequence number the entry must get — the twin’s path: it holds the primary’s numbers, not its own. Refuses anything but head + 1.

pub fn append_raw(&self, kind: u16, meta: &[u8], blob: &[u8]) -> Result<u64>

Append with raw kind and meta bytes. Durable when this returns.

Group commit for concurrent appenders (perf twin-apply-fsync, 2026-10-06): the frame is staged under the writer lock, and its fsync is Log::sync_through’s — taken OUTSIDE the writer lock by one leader for every frame staged so far. Eight request threads that append together pay about one fdatasync between them instead of eight; one appender alone pays exactly what it paid before (one write, one fdatasync, one index write).

pub fn stage(&self, kind: Kind, meta: &Meta, blob: &[u8]) -> Result<u64>

Log::append without waiting for durability — for a frame whose mutation is invisible until a LATER frame publishes it (a control document’s shard blob, published by the ControlDoc frame that moves the pointer): that frame’s fdatasync covers this one, which is earlier in the same file. Refused on a follower, as append is.

pub fn stage_raw(&self, kind: u16, meta: &[u8], blob: &[u8]) -> Result<u64>

Log::append_raw without waiting for durability: the frame is whole in the segment and holds its number; nobody reads it and it is not acknowledged until Log::sync_through (or any later sync) makes it durable. The caller MUST NOT change any state the frame describes before then — the log is written before the state, always.

pub fn sync_through(&self, seq: u64) -> Result<()>

Return once frame seq is durable — making it so if nobody is: a waiter becomes the leader when no leader is in its fsync, writes out the stage, fdatasyncs a duplicate of the segment’s descriptor WITHOUT holding the writer lock (appenders keep staging behind it), then indexes and publishes everything that fsync covered and wakes every waiter. A waiter re-checks the head on EVERY wake-up, so the moment a leader’s fsync covers its frame it returns — it never waits for a later leader’s fsync, and no lock is held across I/O.

pub fn sync(&self) -> Result<u64>

Make a group commit durable and visible (perf twin-apply-fsync, 2026-10-06): every frame committed with Pending::commit_unsynced since the last call is written out, fdatasynced ONCE, indexed with ONE write, and published — readers see it and the head moves to the last of them only now. A twin acknowledges its head to the primary, so what it acknowledges is what this made durable. Answers the head. Nothing pending = no I/O.

pub fn unsynced(&self) -> u64

Frames committed and not yet durable (Log::sync makes them so).

pub fn poisoned_by(&self) -> Option<String>

What poisoned the log, if anything did (Error::Poisoned).

pub fn frame_syncs(&self) -> u64

How many fdatasyncs made frames durable since this log was opened — the leader’s in Log::sync_through and a group’s in Log::sync (a roll, a release and a cut sync too, rarely, and are not counted). What the I/O tests hold to a bound per entry.

pub fn begin( &self, seq: Option<u64>, kind: u16, meta: &[u8], blob_len: u64, ) -> Result<Pending<'_>>

Begin a streaming append for a blob whose bytes arrive in chunks — the twin receiving a large pack over the wire without holding it in memory. The writer lock is held until Pending::commit or Pending::abort; the frame does not exist until commit.

pub fn release(&self) -> Result<()>

Give up the writer’s files so the filesystem under the log can be unmounted — the runtime data-set growth (gunnar-server grow), where PID 1 unmounts /data itself.

MEASURED 2026-09-15 under KVM (growbench R5): the open segment’s .log and .idx were the only descriptors still under /data after the stores closed, and umount answered EBUSY — so every runtime growth on a box carrying the log aborted.

Waits for an in-flight Pending (it holds the writer lock), syncs both files, and parks the handles on /dev/null; readers open per call and are unaffected until the unmount itself. Every write refuses with Error::Released until Log::reacquire. Idempotent.

pub fn reacquire(&self) -> Result<()>

Take the files back after the remount. The open segment must be exactly as long as the writer left it — anything else is a different disk, and is refused by name rather than appended to. Idempotent.

pub fn is_released(&self) -> bool

The files are given up (Log::release).

pub fn read(&self, seq: u64) -> Result<Entry>

Read one frame whole.

pub fn locate_blob(&self, seq: u64) -> Result<Located>

Read a frame’s header and meta, and say where its blob is, so a caller can stream the blob in chunks.

pub fn read_pieces<E: From<Error>>( &self, seq: u64, piece: usize, each: impl FnMut(&Located, &[u8], bool) -> Result<(), E>, ) -> Result<Located, E>

One frame’s blob in pieces of at most piece bytes, never whole — the primary’s half of a twin’s fetch (twin-streaming, 2026-10-06). each gets the frame’s header and meta (Located), one piece, and whether it is the LAST.

The last piece is handed over only after the payload crc checked out, the same check Log::read makes before it answers. A reader that makes the entry visible on last (a twin’s commit) therefore never makes a corrupt one visible: a frame that fails its crc ends in Error::Corrupt with every piece but the last delivered, and the reader drops what it was given. Memory is one piece, whatever the blob’s size.

An erased frame (Kind::Redacted) is one empty last piece and is not checked: its bytes are gone and its crc was over them. A frame with an empty blob checks the crc over its meta and is one empty last piece.

pub fn read_pieces_from<E: From<Error>>( &self, seq: u64, from: u64, piece: usize, each: impl FnMut(&Located, &[u8], bool) -> Result<(), E>, ) -> Result<Located, E>

Log::read_pieces from byte from of the blob on (restore-large-entry, 2026-10-07): the twin’s half of a restore resuming an entry the box being filled holds part of. 0 is Log::read_pieces exactly. Otherwise only the blob from from is read — a resumed entry of gigabytes costs the twin its REST on every call, never a re-read of what was sent, which on a slow volume would eat the call’s budget and stall the restore again near the end — and the crc cannot be checked here (it is over all of it): the last piece carries the frame’s stored crc as Located::payload_crc, unchecked, and the receiver checks it over the entry it put together. from at the blob’s end is one empty last piece; from past it, or on an erased frame, is refused: an InvalidInput I/O error naming both numbers.

pub fn erase(&self, before: u64, repos: &[String]) -> Result<Erased>

Erase every frame before before that carries data of one of repos — the Pack, Refs, LfsPut and LfsDelete frames whose meta names one of them exactly — in place (owner ruling g-nonpay: an account deleted for non-payment is deleted from the log too, on main and on the twin).

Per segment, every matching frame is rewritten as Kind::Redacted: the HEADERS first, made durable with ONE fdatasync (from then on every reader reads those frames as erased, and a crash leaves erased frames, never a frame whose checksum fails), then the blobs’ ranges are punched out of the file (zeros where the filesystem cannot punch), then ONE fsync — two per segment, where it used to be one per frame (R5.1-B). The sequence, the offsets and the index are untouched: a twin’s gap rule, a reader’s seek and the retention floor mean exactly what they meant.

RepoStore frames stay: a replay needs the store’s create and purge to land in order. Frames AT or after before are never touched — a repository created again under the same name after the erase is somebody’s new data. Idempotent: an erased frame is punched again (cheap), so an erase a crash cut between its headers and its punches is finished by the next. Appends do not wait for it (R5.1-B, 2026-10-07): the writer lock is taken only to publish what is staged and to read the plan, and an erase only touches frames below before, all of them published; one erase runs at a time.

pub fn erase_syncs(&self) -> u64

How many fsyncs Log::erase made since this log was opened.

pub fn scan_meta(&self, each: impl FnMut(u64, u16, &[u8])) -> Result<()>

Walk every retained frame’s header and meta, oldest first, without its blob: one open and one sequential buffered read per SEGMENT, each blob skipped by a relative seek inside the buffer, one header array and one meta buffer reused for every frame (T17).

Log::locate_blob opens the segment file once per frame; a scan that runs on every start (gunnar_server::twin::rebuild_audit_trail) cannot pay an open(2) per frame of a long log. Bounded by the head at call time; a frame whose header does not decode, or whose sequence is not the one its position says, is Error::Corrupt.

pub fn range(&self, from: u64, to: u64) -> Result<Vec<Entry>>

Read from..=to whole (bounded by the head).

pub fn truncate_before(&self, seq: u64) -> Result<u64>

Drop every whole segment whose last sequence is below seq. The open segment is never dropped. Returns the new first sequence.

pub fn retain_from(&self, seq: u64) -> Result<u64>

Retain from seq: close the open segment (when it holds anything) so every frame up to the head sits in a closed segment, then drop every whole segment below seq. Returns the new first sequence. seq past the head retains nothing old: the log keeps its numbers and the next append takes head + 1. The operator’s retention door — and the drill’s way to open a retention hole without writing a gigabyte first.

pub fn reset(&self, keep_at_most: u64) -> Result<()>

Start over: drop every segment and begin again at sequence 1 with nothing in the log. The restore door on a fresh box: its own first frames (its minted KEK) make way for the twin’s numbers. Refused when the log holds more than keep_at_most frames — a box with history is not fresh, and a reset would be data loss with a verb on it.

pub fn restart_after(&self, at: u64) -> Result<()>

Start over after at: drop every segment; the log is empty with its head at at and the next append taking at + 1. What a twin does once it has installed a snapshot taken at at — it holds that state, and the log resumes from the next number.

pub fn layout(&self) -> Vec<(u64, u64, u64)>

The segments as retention sees them, oldest first, the open one last: (first, last, bytes) — last is first - 1 for a segment that holds no frame yet. What a byte-bounded retention decides from (2026-10-06, cov-13): only whole segments are ever dropped, so the bytes a from keeps are the sum of the segments at and after the one holding it.

pub fn bytes(&self) -> u64

Bytes on disk across all segments.

Trait Implementations

impl Debug for Log

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.