Struct Reader

Source

pub struct Reader { /* private fields */ }

Expand description

A metadata-first, open-time snapshot of an Acta v0.2 file.

Implementations§

Source §

impl Reader

Source

pub fn open<P: AsRef<Path>>(path: P) -> Result<Self>

Open a file and discover its complete committed schema and data frames.

The file extent is captured before scanning, so later appends do not alter this reader; reopen the path to obtain a newer snapshot. A final frame the file ends inside is an interrupted append: it is excluded from the snapshot and reported by FileMetadata::incomplete_tail. A frame that is present in full but damaged is corruption and fails the open.

Section 6.2 permits a metadata-only open to defer the frame body CRC. This reader deliberately does not: every committed frame is checksummed here, which costs one pass over the file and in exchange makes an interrupted append distinguishable from damage at open time rather than at first read.

Source

pub fn open_with_limits<P: AsRef<Path>>(path: P, limits: Limits) -> Result<Self>

Open a file with explicit frame, metadata, and snapshot limits.

Source

pub fn refresh(&mut self) -> Result<RefreshReport>

Extend this snapshot with frames committed since it was opened or last refreshed.

Refresh starts at this reader’s own committed boundary — its last good offset, expected next sequence, and expected next implicit row ID — and validates every frame it discovers beyond that boundary exactly as Self::open validates a whole file: prefix and prefix CRC, bounded lengths and checked arithmetic, header, payload, trailer, and body CRC, frame type and sequence, schema ID, block metadata, implicit row-ID continuity, and the configured resource limits. Newly discovered blocks are appended to the snapshot in exact file/sequence order.

The snapshot is mutated only after the discovery pass succeeds, so a failed refresh leaves every field and block vector unchanged. A refresh that finds no growth succeeds with zero additions; one that finds only an incomplete next frame succeeds with zero additions and reports incomplete_tail. Repeated refreshes never add the same frame twice, and a no-growth refresh leaves the reported file size unchanged.

A physically incomplete tail is an interrupted append, not corruption, and may become a complete committed frame by the next refresh. A later refresh also accepts a safe Stage 8b repair that shrank only that uncommitted tail back to the previous last_good_offset, and it clears the tail. A file that shrank below the committed boundary is refused with ErrorKind::FileTruncated.

§Refusing another file at the same path

A refresh will not extend this snapshot from a file that is not the one it came from, and it checks that in two ways because neither alone is enough. It compares the path’s current file-system identity against the identity captured at open, which catches a replacement that unlinked and recreated the path. And, because an in-place truncate-and-rewrite keeps that identity while replacing every byte, it re-reads the commit trailer of the last frame this snapshot committed — its length, sequence, body CRC, own CRC, and commit magic — and refuses unless it still matches. On growth it also compares the prologue and schema it re-reads against the ones this snapshot holds. Any mismatch is ErrorKind::FileReplaced, never a silent adoption, and none of it depends on the v0.2 file ID, which the deterministic writer stores as a non-unique zero.

One boundary is worth stating plainly. A snapshot holding no data blocks has only its prologue and schema frame to be recognised by, so a replacement whose prologue and schema are byte-identical is accepted — but such a file agrees with everything the snapshot has exposed, so nothing it already reported can be contradicted.

Refresh never truncates, repairs, or acquires the writer lock.

let mut reader = Reader::open("data.acta")?;
let report = reader.refresh()?;
println!("{} new block(s)", report.blocks_added());

Source

pub fn tail(&mut self) -> Tail<'_>

Start a live tail over this reader.

The tail begins after the blocks this snapshot already holds and polls the path for newly committed frames without reopening or rescanning the committed prefix. Existing blocks remain available through Self::scan.

A poll returns None when nothing is committed yet, which is never the end of the stream, so a tail is followed until the caller decides to stop rather than until it runs out — that is why this loop is not a while let.

let mut reader = Reader::open("data.acta")?;
let mut tail = reader.tail().project(["value"])?;
while keep_following() {
    match tail.poll_next()? {
        // one newly committed matching block per poll
        Some(batch) => println!("{} new row(s)", batch.row_count()),
        // nothing committed yet; the caller chooses how long to wait
        None => wait_a_while(),
    }
}

Source

pub fn schema(&self) -> &Schema

The immutable schema reconstructed from the schema frame.

Source

pub fn file_metadata(&self) -> &FileMetadata

File-level metadata captured at open time.

Source

pub fn blocks(&self) -> &[BlockMetadata]

Complete data blocks in file/sequence order.

Source

pub fn total_rows(&self) -> u64

The checked sum of rows in all complete blocks.

Source

pub fn scan(&self) -> Scan<'_> ⓘ

Lazily decode the complete blocks in this snapshot in file order.

The iterator reads one block only when its item is requested and yields Result<crate::RecordBatch> so decode and I/O failures are reported during iteration. The scan is isolated from later appends: it uses the file extent and committed block list captured by this reader when it was opened.

By default every column is returned in schema order. Scan::project restricts and reorders the columns, and Scan::primary_range restricts the rows, pruning whole blocks by their stored bounds before reading them. Both are configured against the captured schema and fail immediately rather than during iteration.

Source

pub fn read_block(&self, index: usize) -> Result<RecordBatch>

Decode one complete data block in schema column order.

The index is the zero-based position returned by Self::blocks, which is also file and sequence order; it is not the frame sequence number, which starts at one. An index past the last block is a caller mistake rather than a defect in the file, so it is reported as ErrorKind::InvalidArgument.

Decoding uses the file extent captured by Self::open, so a later append cannot become part of this snapshot.