Expand description
Pure range chunking shared by faucet backfill and the partition: block
(#479). Boundary parsing, timezone/DST-correct time windows, integer-range
splitting, and offset/limit splitting. No I/O.
Both consumers are CLI-layer — the substitution that uses these chunks walks
config strings, which no connector crate participates in — so this lives here
rather than in faucet-core. That also avoids making chrono non-optional
and adding chrono-tz to core, neither of which a connector author needs.
The time half was moved here verbatim from backfill::plan, which now
re-exports it, so faucet backfill keeps byte-identical planning (and its
existing tests keep passing against the moved code).
§Bounds are the correctness hazard
An integer range can be split two ways, and picking wrong is silent data
loss, not an error. With chunk_size: 10000 from 0:
Bounds | chunk 1 | chunk 2 | emitted end |
|---|---|---|---|
Inclusive | [0, 9999] | [10000, 19999] | 9999 |
HalfOpen | [0, 10000) | [10000, 20000) | 10000 |
Half-open chunks against an API whose upper bound is inclusive fetch record 10000 twice; inclusive chunks against an exclusive API never fetch record 9999. Neither surfaces as a failure, which is why the config field has no default — the user has to state which their source is.
Structs§
- IntChunk
- One independent slice of an integer range.
- Offset
Chunk - One
offset/limitslice of a countable result set. - Time
Chunk - One independent, resumable slice of a time range.
Enums§
- Bounds
- Whether a chunk’s upper edge is included. See the module docs — this has no default on purpose.
- Window
Step - How a window advances the cursor.
Constants§
- MAX_
UNITS - Hard ceiling on planned chunks — a tiny window over a huge range is a config error, not a workload.
- WARN_
UNITS - Above this many chunks a loud warning is emitted (but planning proceeds).
Functions§
- parse_
boundary - Parse a range boundary: RFC3339 (
2026-06-01T00:00:00Z) or a bare date (2026-06-01, interpreted as midnight intz). A date that falls in a DST gap resolves to the earliest valid instant. - parse_
window - Parse a window duration:
45s,30m,6h,1d,1w(or a bare integer = seconds). Must be positive. - plan_
int_ chunks - Split
[from, to](or[from, to)perbounds) into contiguous chunks of at mostchunk_sizevalues. - plan_
offset_ chunks - Split a result set of
totalrows intooffset/limitchunks. - plan_
windows - Chunk
[from, to)into contiguous half-open windows ofwindow(the last window truncated atto).window: None= the whole range as one unit. Window arithmetic is absolute (instants), so units never gap or overlap — including across DST transitions; boundaries are re-rendered intzso${now.*}tokens see local wall-clock time.