Skip to main content

scan_frames

Function scan_frames 

Source
pub fn scan_frames(bytes: &[u8]) -> PushLogScan
Expand description

Read forward over concatenated frames, stopping at the first that does not parse. Never errors on a torn tail — that is the expected state after a crash and the whole reason the framing was chosen.

§Why “did the reader error?” is not enough

Arrow’s StreamReader treats an unexpected EOF as a clean end of stream, so a frame truncated inside its record-batch message parses without error and simply yields no batch. Trusting the reader’s verdict alone therefore accepted a torn push as a complete-but-empty frame and advanced past it — the torn bytes were silently swallowed, torn_tail_bytes reported 0, and a crashed push was indistinguishable from a clean log.

Two positive checks close that, and both must hold for a frame to count:

  1. it yielded exactly one RecordBatch — the format’s invariant is one push, one batch, so zero batches is a truncation and two is not our frame;
  2. its last eight bytes are [IPC_EOS] — proof the writer reached finish() rather than the reader reaching the end of the file.