Expand description
Completeness reconciliation (#502).
An opt-in post-run guard against silent truncation: after a successful
root run, fetch an authoritative row count for the same data (a count(*)
query, an OData $count, …) and compare it to the number of rows this run
wrote. A shortfall beyond tolerance_pct fails the run — the point being
that a half-read source must not quietly replace good data with less
(especially under write_mode: overwrite, #492/#494).
The authoritative count comes from a small count-probe source the user
configures (any faucet source that yields the count as a single value/row),
so reconciliation works for any backend without a per-connector count()
capability. Pure evaluation lives in evaluate / extract_count; the
CLI executor runs the probe post-run.
Structs§
- Count
Probe - A source that yields the authoritative count.
- Reconcile
Spec - Top-level
reconcile:block.
Functions§
- evaluate
- Compare rows written against the authoritative count.
Ok(())whenwritten >= authoritative * (1 - tolerance_pct/100), elseErrwith a message naming the shortfall. Pure. - extract_
count - Extract the authoritative count from a probe’s returned records. Uses the
named
fieldwhen given, else the first numeric field of the first record. Pure. - run
- Run the count probe and reconcile it against
written. Builds the probe source through the registry (so any connector works and sharedauth: { ref }resolves), drains it, extracts the count, and evaluates. ReturnsFaucetError::Sourceon a shortfall so the caller fails the run.