Skip to main content

Module partition

Module partition 

Source
Expand description

Generic range partitioning (#479) — split one matrix row into N independent invocations, each scoped to a chunk of a range via ${partition.*} tokens.

This is source-agnostic by construction: substitution walks the string leaves of a connector config, so a REST URL, a SQL WHERE, an object prefix and a Mongo filter all work with no connector code. See the plan module for the mechanism and spec for why the kinds are a tagged enum.

Re-exports§

pub use plan::PartitionChunk;
pub use plan::plan;
pub use plan::references_partition;
pub use plan::substitute;
pub use probe::needs_probe;
pub use probe::resolve_bounds;
pub use spec::BoundProbe;
pub use spec::CountBound;
pub use spec::IntBound;
pub use spec::PartitionSpec;

Modules§

plan
Turn a PartitionSpec into concrete chunks and substitute their ${partition.*} tokens into connector configs (#479).
probe
Discover a partition bound by running a source once (#479).
spec
Serde config types for the partition: block (#479).

Functions§

resolve_config_bounds
Resolve every discoverable partition bound in cfg in place (#479).