Skip to main content

Module range_stream

Module range_stream 

Source
Expand description

Range-indexed WAL streaming and per-range catch-up (issue #992).

The physical WAL stays a single sequential append log — there is no per-range physical file. Range replication is a filtered view over that one log: every derived ChangeRecord carries its range identity, owning term, and ownership epoch (issue #991), so a data member can index and route records by range without splitting the log.

This module supplies the transport-agnostic primitives a range replica or a move-range target uses to ride that filtered view:

  • RangeStreamPosition — the per-range resume point and authority watermark a follower persists. Catch-up restarts from this position rather than from the global stream head.
  • classify_range_record / plan_range_catchup — gate the shared stream down to exactly the records a range should apply: only records stamped for the range, only those past the resume LSN, only those whose term and ownership epoch clear the range authority (stale owners fenced).
  • RangeStreamProgress / RangeProgressTracker — independent lag and progress per range over the one physical stream, enough to reason about per-range failover eligibility later (issue #987 parent).

The actual on-disk apply (the LSN state machine, payload hashing) stays in reddb-server; this crate only describes the routing/gating contract.

Structs§

RangeCatchupPlan
The result of filtering a slice of the shared logical stream down to one range’s catch-up work. apply holds indices into the input slice in ascending LSN order; resume is the position advanced past every applied record (persist it to make catch-up resumable); rejected lists the fenced records.
RangeProgressTracker
Tracks streaming progress for many ranges over the one physical WAL. A data member feeds every derived record through index_record to maintain each range’s primary frontier without splitting the log, and notes shipped/applied LSNs per range. Every range advances independently, so one lagging range does not skew another’s lag or failover eligibility.
RangeStreamPosition
The per-range resume position and authority watermark a range follower persists. A range replica restarting catch-up hands the primary this position so streaming resumes from applied_lsn for the range instead of replaying the whole shared WAL, and so the follower keeps fencing records from a deposed owner.
RangeStreamProgress
Independent streaming progress for one range over the shared physical WAL.
RangeStreamReject
A record that range catch-up refused, with the reason. Surfaced (rather than silently dropped) so the caller can count fenced records and tell a stale owner apart from a quiet range.

Enums§

RangeStreamDecision
Why a single record was routed the way it was during range catch-up.

Functions§

classify_range_record
Route a single record relative to a range’s resume position. Routing order: other-range first (cheapest, most common), then already-applied, then the authority fence — so a record we have already applied is skipped without being treated as a fence violation, while a fresh stale-owner write is rejected.
plan_range_catchup
Filter a slice of the shared logical stream (ascending by LSN) into the catch-up plan for a single range, resuming from position. Only records stamped for position.range_id, past its resume LSN, and clearing its authority fence are selected; the returned resume position has been advanced past them so a follower can persist it and continue.