Skip to main content

Module stream

Module stream 

Source
Expand description

SingerSource — the Source implementation that bridges a Singer tap.

This is the only place that drives the tap subprocess. Page assembly rules (v0, single-stream):

  • RECORD for the configured stream → buffered.
  • STATE → (if flush_on_state) flush the buffer as a page whose bookmark is the STATE value; otherwise remember it as the pending checkpoint.
  • buffer reaches the batch_size hint → flush a page with bookmark: None (no STATE covers these rows yet; they re-emit from the last checkpoint on resume and an idempotent sink dedups them).
  • clean EOF → flush any trailing buffer + pending checkpoint.
  • non-zero exit / idle timeout / malformed-fail → terminate and error without committing the trailing un-checkpointed buffer.

Structs§

SingerSource
A source that runs a Singer tap and adapts its output into faucet records.