pub fn scan_frames(bytes: &[u8]) -> PushLogScanExpand description
Read forward over concatenated frames, stopping at the first that does not parse. Never errors on a torn tail — that is the expected state after a crash and the whole reason the framing was chosen.
§Why “did the reader error?” is not enough
Arrow’s StreamReader treats an unexpected EOF as a clean end of stream, so
a frame truncated inside its record-batch message parses without error and
simply yields no batch. Trusting the reader’s verdict alone therefore
accepted a torn push as a complete-but-empty frame and advanced past it —
the torn bytes were silently swallowed, torn_tail_bytes reported 0, and a
crashed push was indistinguishable from a clean log.
Two positive checks close that, and both must hold for a frame to count:
- it yielded exactly one
RecordBatch— the format’s invariant is one push, one batch, so zero batches is a truncation and two is not our frame; - its last eight bytes are [
IPC_EOS] — proof the writer reachedfinish()rather than the reader reaching the end of the file.