Struct PlanBuilder 

pub struct PlanBuilder(/* private fields */);

Available on crate feature declarative-plans only.

A Plan under construction. See the module docs.

Implementations

impl PlanBuilder

pub fn scan_parquet( files: impl IntoIterator<Item = impl Into<ScanFile>>, file_constant_columns: &[&str], schema: impl Into<SchemaRef>, ) -> DeltaResult<Self>

A Parquet scan source over files producing rows matching schema.

file_constant_columns names the per-file-constant output columns (see ScanParquet); pass &[] when there are none. An empty files yields the absent relation.

Produces an error when any file’s constant count differs from file_constant_columns’s length, or a file_constant_columns entry is absent from schema.

pub fn scan_json( files: impl IntoIterator<Item = impl Into<ScanFile>>, file_constant_columns: &[&str], schema: impl Into<SchemaRef>, ) -> DeltaResult<Self>

A newline-delimited JSON scan source over files producing rows matching schema. See ScanJson. Exhibits same behaviour as Self::scan_parquet.

pub fn values( schema: impl Into<SchemaRef>, rows: Vec<Vec<Scalar>>, ) -> DeltaResult<Self>

Inline literal rows. See Values for the row encoding. Empty rows yields the absent relation.

Produces an error when any row’s width differs from schema’s top-level field count.

Example

let schema = Arc::new(StructType::try_new([StructField::not_null("id", DataType::INTEGER)])?);
let plan = PlanBuilder::values(schema, vec![vec![1.into()], vec![2.into()]])?.build()?;

pub fn filter(self, predicate: impl Into<PredicateRef>) -> DeltaResult<Self>

Keep rows where predicate holds. Output schema is unchanged. See Filter.

Produces an error when predicate references a column absent from the input schema.

Example

let schema = Arc::new(StructType::try_new([StructField::not_null("id", DataType::INTEGER)])?);
let plan = PlanBuilder::values(schema, vec![vec![1.into()], vec![2.into()]])?
    .filter(col!("id").is_not_null())?
    .build()?;

pub fn project( self, expr: impl Into<ExpressionRef>, schema: impl Into<SchemaRef>, ) -> DeltaResult<Self>

Project self through expr into rows of the caller-declared schema. expr must be a struct constructor / patch matching schema. See Project.

Produces an error when expr references a column absent from the input schema. References inside a StructPatch are validated by ProjectionStructPatchBuilder when the patch is

Example

let input = Arc::new(StructType::try_new([
    StructField::not_null("id", DataType::INTEGER),
    StructField::nullable("name", DataType::STRING),
])?);
let out = Arc::new(StructType::try_new([StructField::not_null("id", DataType::INTEGER)])?);
// Keep only `id`.
let plan = PlanBuilder::values(input, vec![vec![1.into(), "a".into()]])?
    .project(Expression::struct_from([col!("id")]), out)?
    .build()?;

pub fn project_patch( self, patch: impl FnOnce(ProjectionStructPatchBuilder<'_>) -> ProjectionStructPatchBuilder<'_>, ) -> DeltaResult<Self>

Project self by editing its columns. edit receives a ProjectionStructPatchBuilder rooted at self’s schema and records field edits (replace/drop/append); the lowered patch and the resulting output schema feed the Project node together. See Project.

Produces an error when the patch fails to build – e.g. a replaced or dropped field is absent from the input schema, or a nested path does not resolve.

pub fn load(self, load: Load) -> DeltaResult<Self>

Read data files named by self’s rows. Output schema is load.schema. See Load.

Produces an error when a file_meta/deletion-vector column is absent from the input schema, or a file_constant_columns entry is absent from the input (its broadcast source) or from load.schema (the output).

pub fn aggregate( self, aggregate: impl TryInto<Aggregate, Error = Error>, ) -> DeltaResult<Self>

Aggregate self into aggregate (build one with Aggregate::group_by). The output schema is the group keys followed by the aggregate columns. See Aggregate.

Over an absent input the result is group-arity dependent: a grouped aggregate yields zero groups and so is absent; a global aggregate (no group keys) still yields one row, so it aggregates an empty Values input and lets the engine produce that row.

Produces an error when building aggregate fails: a group key or an aggregate’s operand column is absent from its input schema, or two output columns would share a name.

pub fn aggregate_by( self, keys: impl CollectInto<Vec<ColumnName>>, aggs: impl FnOnce(AggregateBuilder) -> AggregateBuilder, ) -> DeltaResult<Self>

Aggregate self, grouped by keys. aggs receives an AggregateBuilder rooted at self’s schema and adds the aggregate columns (see AggregateBuilder::max, …). Mirrors Self::aggregate but spares the caller from naming the input schema, just as Self::project_patch does for Self::project. See Aggregate.

Produces an error under the same conditions as Self::aggregate.

Example

let schema = Arc::new(StructType::try_new([
    StructField::not_null("id", DataType::INTEGER),
    StructField::nullable("version", DataType::LONG),
])?);
// Latest `version` per `id`.
let plan = PlanBuilder::values(schema, vec![vec![1.into(), 7i64.into()]])?
    .aggregate_by([column_name!("id")], |a| a.max(column_name!("version")))?
    .build()?;

pub fn semi_join( self, build: PlanBuilder, probe_keys: impl IntoIterator<Item = ColumnName>, build_keys: impl IntoIterator<Item = ColumnName>, ) -> DeltaResult<Self>

Semi join: emit the self (probe) rows that have a match in build on the join keys. Output schema mirrors self. Inputs are recorded as [probe, build]. See SemiJoin.

Keys are columns (e.g. column_name!), matched pairwise.

Produces an error when the probe and build key counts differ, or a probe/build key is absent from its input schema.

Example

let schema = Arc::new(StructType::try_new([StructField::not_null("id", DataType::INTEGER)])?);
let probe = PlanBuilder::values(Arc::clone(&schema), vec![vec![1.into()], vec![2.into()]])?;
let allow = PlanBuilder::values(schema, vec![vec![2.into()]])?;
// Probe rows whose `id` appears in `allow`.
let plan = probe.semi_join(allow, [column_name!("id")], [column_name!("id")])?.build()?;

pub fn anti_join( self, build: PlanBuilder, probe_keys: impl IntoIterator<Item = ColumnName>, build_keys: impl IntoIterator<Item = ColumnName>, ) -> DeltaResult<Self>

Anti join: emit the self (probe) rows that have no match in build on the join keys. An inverted SemiJoin; otherwise as Self::semi_join.

pub fn union_all( inputs: impl IntoIterator<Item = PlanBuilder>, ) -> DeltaResult<Self>

Unordered bag union of inputs into a UnionAll. All inputs must share the same schema; absent inputs are dropped, and a lone present input is forwarded unchanged.

Produces an error when inputs is empty, or two inputs have differing schemas.

Example

let schema = Arc::new(StructType::try_new([StructField::not_null("id", DataType::INTEGER)])?);
let a = PlanBuilder::values(Arc::clone(&schema), vec![vec![1.into()]])?;
let b = PlanBuilder::values(schema, vec![vec![2.into()]])?;
let plan = PlanBuilder::union_all([a, b])?.build()?;

pub fn build(&self) -> DeltaResult<Plan>

Linearize the DAG reachable from self into a Plan. An absent relation builds to a single empty Values node carrying its schema, so the result is always a runnable plan.

pub fn build_opt(&self) -> DeltaResult<Option<Plan>>

Like Self::build, but yields None for an absent relation instead of an empty plan.

Trait Implementations

impl Clone for PlanBuilder

fn clone(&self) -> PlanBuilder

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PlanBuilder

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.