Struct EpochStaging 

pub struct EpochStaging { /* private fields */ }

Commit-gated output staging for an exactly-once sink that has no external transaction of its own (the Arrow file sink, the Flight sink) — the software analogue of the Kafka sink’s producer transaction. on_data buffers into the open epoch; on_barrier seals it (pre-commit); commit releases every sealed epoch up to the committed one for the sink to make visible. Nothing is visible before its epoch commits, so a restart from the last committed checkpoint re-produces the same rows (deterministic keys) and each committed row appears exactly once. Uncommitted staged rows are dropped on stop — a replay re-emits them. (At-least-once sinks skip this and emit in on_data, unchanged.)

Implementations

impl EpochStaging

pub fn push(&mut self, batch: RecordBatch)

Buffer a batch into the epoch currently open (the one the next barrier will seal).

pub fn seal(&mut self, epoch: u64)

Seal the open epoch at its barrier: its batches become committable under epoch.

pub fn commit(&mut self, epoch: u64) -> Vec<RecordBatch>

Release every sealed epoch <= epoch (commits are monotonic) for the sink to make visible.

pub fn drain_all(&mut self) -> Vec<RecordBatch>

Flush EVERYTHING still staged (sealed epochs in order, then the open tail) for a clean stop — the sink calls this in on_eos. A clean Eos arrives only when every upstream source ended cleanly (a crash never delivers it), and a bounded run reaching Eos has drained without a final barrier over its tail; so this is the tail’s one committed emission. (An unbounded clean stop commits its tail through the final coordinated epoch before Eos, so this finds nothing.)

Trait Implementations

impl Default for EpochStaging

fn default() -> EpochStaging

Returns the “default value” for a type.