Struct ParquetTableSpec 

pub struct ParquetTableSpec {
    pub name: String,
    pub path: String,
    pub primary_key: Vec<String>,
}

A parquet table to plan against, and what is known to be true about it.

primary_key is an informational constraint, in the sense Spark and Databricks use the word (RELY): the engine does not verify it, it trusts it. Declaring a key that does not hold changes results, exactly as a wrong RELY constraint does there. It is empty by default, so a caller that says nothing gets the previous behaviour.

Why the engine wants to be told

DataFusion already derives FunctionalDependencies from a provider’s constraints, and optimize_projections already uses get_required_group_by_exprs_indices to shrink a GROUP BY to the minimal functionally-equivalent subset. All of that machinery is live and does nothing here, because a parquet file carries no key and every table we register declares none — so the dependency set is always empty.

What this does and does NOT buy — read before assuming

DataFusion’s rule keeps (columns the parent requires) ∪ (minimal FD subset). So a declared key removes a column from a GROUP BY only when nothing downstream selects it. That is a real and common shape — GROUP BY a, b projecting only a — and it is what a_declared_primary_key_shrinks_the_group_by proves.

It is not TPC-H q10’s shape. q10 selects all seven grouped columns, so the parent requires them and no key declaration can prune them. Measured 2026-07-31: rewriting q10 to carry only the key is worth 14.8x (1784.6 s → 120.9 s; the orders⋈customer stage alone 8,968 → 38.9 task-seconds), but capturing that needs late materialisation — aggregate on the key, take the top N, then re-join for the display columns. DataFusion has no such rule and neither do we. Declaring a key is a precondition for writing one, not a substitute.

The 230x on that stage is superlinear in the columns rather than the bytes: per-row string handling (hashing and copying ~227 B rows), which is why neither bandwidth nor fetch concurrency moved it (q10-dist-s2-is-the-whole-query).

Fields

name: String

Name the query refers to the table by.

path: String

Single parquet file or a directory dataset; local or object storage.

primary_key: Vec<String>

Columns that jointly form a primary key, if the caller declares one.

Implementations

impl ParquetTableSpec

pub fn new(name: impl Into<String>, path: impl Into<String>) -> Self

A table with nothing declared about it.

pub fn with_primary_key<I, S>(self, columns: I) -> Self

where I: IntoIterator<Item = S>, S: Into<String>,

Declare a (possibly composite) primary key.

Trait Implementations

impl Clone for ParquetTableSpec

fn clone(&self) -> ParquetTableSpec

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ParquetTableSpec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.