Expand description
Turn a PartitionSpec into concrete chunks and
substitute their ${partition.*} tokens into connector configs (#479).
Substitution walks every string leaf of a config Value, exactly like
backfill::plan::substitute_unit_tokens — which is why this works for any
source without a line of connector code. A REST URL, a SQL WHERE, an object
prefix and a Mongo filter are all just strings.
§Injection
Tokens land inside SQL and JSON strings, where substitute_context is
documented as unsafe. What makes it safe here is that every rendered value is
faucet-generated and type-constrained — an i64, a u64, or an RFC3339
timestamp formatted by chrono — never user data and never a passthrough of
anything a remote system said. plan() is the only place token values are
produced, and it produces them from typed fields — an i64 start/end, a
u64 offset/limit, a chrono-formatted timestamp.
Structs§
- Partition
Chunk - One planned chunk: a stable id plus the tokens its invocation substitutes.
Functions§
- bounds_
label - Bounds label for log lines.
- has_
open_ ended - The
endpredicate for an open-ended final chunk is the caller’s problem — this reports whether one was planned soexpandcan require the config to handle it. - plan
- Plan every chunk for
spec. - references_
partition - Whether a serialized config references any
${partition.*}token. - substitute
- Substitute
${partition.*}tokens in every string leaf ofvalue.