Skip to main content

Module lazy_reader

Module lazy_reader 

Source
Expand description

Lazy WAL reader: reads headers without payload for selective replay.

Unlike WalReader which reads every payload into a Vec<u8>, this reader reads only the 30-byte header first. The caller inspects the header (record_type, vshard_id, lsn) and decides whether to read or skip the payload.

This is critical for startup replay performance: with 100M timeseries records, a vector core can skip TS payloads (potentially GBs) by seeking forward instead of allocating and reading.

§Usage

let mut reader = LazyWalReader::open(path)?;
while let Some(header) = reader.next_header()? {
    if header.record_type == RecordType::VectorPut as u32 {
        let payload = reader.read_payload(&header)?;
        // process vector record
    } else {
        reader.skip_payload(&header)?;
    }
}

Structs§

LazyWalReader
Lazy WAL reader that separates header reading from payload reading.

Functions§

replay_all_segments_lazy
Replay all WAL segments in a directory with lazy reading.
replay_segment_lazy
Open a WAL segment for lazy reading and iterate with a callback.