#[non_exhaustive]pub enum SourceEvent<L> {
LanesAssigned(Vec<L>),
LanesAdded(Vec<L>),
LanesRevoked {
lanes: Vec<LaneId>,
barrier: DrainBarrier,
},
LanesRetired {
lanes: Vec<LaneId>,
},
Idle,
CommitReady {
partitions: Vec<PartitionId>,
},
Drained,
}Expand description
Control-plane event returned by Source::poll_events.
Variants (Non-exhaustive)§
This enum is marked as non-exhaustive
LanesAssigned(Vec<L>)
New lanes were assigned; the runtime distributes them across pipeline threads. The source bumps its assignment epoch first.
LanesAdded(Vec<L>)
Additional lanes join the current assignment epoch; existing
lanes are untouched and their in-flight batches keep resolving.
Coordinated sources emit this for incremental split gains so a
routine gain never drains flowing lanes (contrast
SourceEvent::LanesAssigned, whose eager-rebalance contract
replaces the full lane set). Lane ids must be new — never reuse an
id from this source’s lifetime — and each lane’s partition must be
fresh, not one revoked earlier in the epoch.
LanesRevoked
Lanes are being revoked. The runtime trips the DrainBarrier for
the owning threads, which stop the lanes, flush in-flight records,
and arrive; the source completes the revocation (final synchronous
commit included) only after DrainBarrier::wait returns.
Fields
barrier: DrainBarrierBarrier the owning pipeline threads arrive at once drained.
LanesRetired
Lanes whose work is finished are leaving the assignment: their
input is fully delivered, acknowledged, and committed (e.g. a
coordinated split whose terminal progress reached the store), so
by contract nothing unflushed or uncommitted can exist behind
them. The runtime removes them without a drain barrier — pure
bookkeeping, no pipeline stall. Sources must use
SourceEvent::LanesRevoked instead whenever any in-flight data
or uncommitted acknowledgement may remain.
Idle
Nothing happened within the timeout.
CommitReady
A hint that a commit outside the periodic tick is worthwhile now
for the named partitions: their lanes have decided end-of-input,
and the source cannot finalize their unit of work (e.g. complete a
coordinated split, freeing its working-set slot) until the acked
watermark reaches it through Source::commit. The runtime responds
by briefly tightening its commit cadence for those partitions
only — flowing partitions keep the periodic tick — until their
acks quiesce or one commit interval elapses. Purely a latency
optimization: correctness never depends on it, and sources that
never emit it get the periodic cadence.
Fields
partitions: Vec<PartitionId>Partitions whose final acks are worth chasing.
Drained
The source has permanently exhausted its input: every lane has
yielded its final batch and will only ever return Ok(None) again.
Bounded sources (backfills) emit this to request a graceful drain;
the runtime flushes chains, drains sinks, runs a final synchronous
commit, and exits with ExitState::Completed.
Contract: a source must not report Drained while any lane still
holds unemitted data — a lane’s exhaustion may only be decided by a
poll that returned Ok(None) after its final batch was consumed
(the poll→push→poll sequencing on the owning thread then guarantees
the final batch was fully pushed downstream). Emitting Drained is
idempotent: sources should keep returning it once drained.
Unbounded sources never emit it.