Struct LocalParquetSource 

pub struct LocalParquetSource {

    pub files: Vec<LocalParquetFile>,
    pub schema: SchemaRef,
    pub output_schema_override: Option<SchemaRef>,
    pub read_columns: Vec<String>,
    pub read_column_file_indices: Vec<usize>,
    pub projected_output_columns: Vec<String>,
    pub projected_output_column_source_refs: Vec<SourceColumnRef>,
    pub pure_filter_columns: Vec<String>,
    pub pure_filter_column_source_refs: Vec<SourceColumnRef>,
    pub partition_columns: Vec<PartitionColumn>,
    pub missing_columns: Vec<MissingColumn>,
    pub scan_limit: Option<usize>,
    pub reader_predicate: Option<Expr>,
    pub max_rows: Option<usize>,
    pub unique_key_columns: Vec<BTreeSet<String>>,
    pub diagnostics: SourceDiagnostics,
    /* private fields */
}

Frontend-neutral local Parquet scan contract.

Carries files, emitted schema, column roles, reader predicate, and IO diagnostics for native source execution.

Fields

files: Vec<LocalParquetFile>

Local Parquet files to scan, including per-file metadata and pruning.

schema: SchemaRef

Schema of the emitted source output. Read file columns come first, then partition columns.

output_schema_override: Option<SchemaRef>

Optional schema exposed to the query plan when execution transforms the physical read frame before yielding source output.

read_columns: Vec<String>

Physical Parquet columns that the engine must read.

These columns are emitted by the source. They include projected file columns and file columns needed only by source-adjacent filters. Partition columns are excluded because partition values come from file metadata and are emitted after read_columns.

read_column_file_indices: Vec<usize>

File-schema indices for read_columns.

projected_output_columns: Vec<String>

Source columns expected to survive source-adjacent filters/projections.

projected_output_column_source_refs: Vec<SourceColumnRef>

Source references for projected_output_columns.

pure_filter_columns: Vec<String>

Columns emitted only so source-adjacent native filters can evaluate.

These should be removed by a later native projection when they are not needed downstream.

pure_filter_column_source_refs: Vec<SourceColumnRef>

Source references for pure_filter_columns.

partition_columns: Vec<PartitionColumn>

Synthetic source columns populated from file partition values.

missing_columns: Vec<MissingColumn>

Nullable logical source columns materialized as typed null columns.

scan_limit: Option<usize>

Source-level scan limit applied before source-adjacent native filters.

reader_predicate: Option<Expr>

Reader-level Parquet predicate applied by the cuDF reader before source-local native filters.

This predicate is expressed over read_columns; every referenced column must resolve to a physical file column that is present in the read projection. Partition and missing-null columns are not reader-visible.

max_rows: Option<usize>

Trusted source cardinality upper bound after source-level scan limit and reader predicate have been applied.

unique_key_columns: Vec<BTreeSet<String>>

Source-owner unique keys expressed in emitted physical column names.

diagnostics: SourceDiagnostics

Source capability and rejection details captured during lowering.

Implementations

impl LocalParquetSource

pub fn from_files( files: Vec<LocalParquetFile>, schema: SchemaRef, read_columns: Vec<String>, ) -> Self

Builds a source from fully described local Parquet files.

schema must describe the source node’s emitted schema. By default, every read file column is also marked as projected output metadata and no partition columns or pure filter columns are represented.

pub fn new( path: impl Into<PathBuf>, schema: SchemaRef, read_columns: Vec<String>, ) -> Self

Declares a single-file source with the supplied output schema and read columns.

pub fn with_unique_key_columns( self, unique_key_columns: Vec<BTreeSet<String>>, ) -> Self

Attaches sets of columns proven unique by the caller. No data scan is performed to verify these claims.

pub fn zero_column_count(files: Vec<LocalParquetFile>) -> Self

Builds the metadata-only representation for count-only Parquet scans.

Zero-column count sources read no physical or partition columns. They use ZERO_COLUMN_COUNT_COLUMN as the synthetic output field and mark the diagnostics as represented. The caller must have proven that the shape is safe to answer from file or row-group metadata.

pub fn count_star_probe( files: Vec<LocalParquetFile>, read_schema: SchemaRef, read_columns: Vec<String>, ) -> Self

Builds a COUNT(*) fallback source that reads a physical row-presence column but emits one Int64 row-count weight per read unit.

pub fn with_column_roles( self, projected_output_columns: Vec<String>, pure_filter_columns: Vec<String>, ) -> Self

Sets projected and filter-only logical column roles.

This does not change schema; it only records which emitted source columns are expected to survive source-adjacent filtering/projection and which emitted columns are only filter inputs.

Source references are inferred from read_columns and partition_columns.

pub fn with_column_source_refs( self, read_column_file_indices: Vec<usize>, projected_output_column_source_refs: Vec<SourceColumnRef>, pure_filter_column_source_refs: Vec<SourceColumnRef>, ) -> Self

Sets explicit source references for read, projected, and filter-only columns.

Use this when frontend scan metadata already resolved logical columns to file-schema positions or synthetic partition columns.

pub fn with_column_file_indices( self, read_column_file_indices: Vec<usize>, projected_output_column_file_indices: Vec<usize>, pure_filter_column_file_indices: Vec<usize>, ) -> Self

Sets explicit file-schema indices for column roles with no partition columns.

pub fn with_output_schema_override(self, output_schema: SchemaRef) -> Self

Sets the schema exposed to downstream query-plan nodes.

This is used when the physical Parquet read schema contains file-local names, but the source output has already been mapped back to logical table names by explicit output-column refs. Non-count-probe callers must still include every output override field in schema, because execution schema coercion and source-contract lowering operate on that execution source schema.

pub fn with_file_column_count(self, file_column_count: usize) -> Self

Sets the total physical column count in the source Parquet file schema.

pub fn with_partition_columns( self, partition_columns: Vec<PartitionColumn>, ) -> Self

Replaces the partition columns, which are represented as synthetic source columns.

pub fn with_missing_columns(self, missing_columns: Vec<MissingColumn>) -> Self

Replaces the missing columns, which are represented as typed null source columns.

pub fn with_scan_limit(self, scan_limit: Option<usize>) -> Self

Sets an order-insensitive source-level scan limit.

pub fn with_max_rows(self, max_rows: Option<usize>) -> Self

Sets a trusted source cardinality upper bound.

pub fn with_reader_predicate(self, predicate: Expr) -> Self

Sets a reader-level Parquet predicate.

Callers should only set predicates for expressions accepted by Self::reader_predicate_is_supported. Validation also checks this before execution.

pub fn reader_predicate_is_supported( &self, predicate: &Expr, ctx: AnalysisContext<'_>, ) -> bool

Returns true if predicate can be lowered to the cuDF Parquet reader AST using this source’s read-column projection.

pub fn reader_predicate_supported_expression( &self, predicate: &Expr, ctx: AnalysisContext<'_>, ) -> Option<Expr>

Returns a reader-level predicate expression accepted by the cuDF Parquet reader contract.

Most accepted predicates are returned unchanged. Fixed ASCII prefix predicates of the form SUBSTRING(column FROM 1 FOR n) IN (...) are rewritten to equivalent lexicographic string ranges because the cuDF reader AST can express string comparisons but not string slicing.

pub fn reader_predicate_requires_source_contract( &self, predicate: &Expr, ) -> bool

Returns true when the reader predicate needs per-source-chunk metadata to lower correctly.

pub fn refresh_diagnostics(&mut self)

Recomputes source diagnostics from current files and column roles, preserving reader-predicate and metadata-only count dispositions.

pub fn output_schema(&self) -> &SchemaRef

Returns the schema this source exposes to the native plan graph.

pub fn is_zero_column_count_source(&self) -> bool

Returns true when this source is the metadata-only zero-column count form.

pub fn is_count_star_probe_source(&self) -> bool

Returns true when this source reads a carrier column only to emit count weights.

pub fn file_count(&self) -> usize

Number of file entries represented by the source.

pub fn read_column_count(&self) -> usize

Number of physical columns selected for decoding.

pub fn projected_output_column_count(&self) -> usize

Number of source columns expected to remain after source-adjacent native filtering/projection.

pub fn pure_filter_column_count(&self) -> usize

Number of emitted source columns used only by source filter evaluation.

pub fn selected_row_group_count(&self) -> usize

Total selected row groups across files, or zero when unknown.

pub fn exact_selected_row_count(&self) -> Option<u64>

Returns the exact number of rows represented by the selected Parquet row groups, when every selected group has footer-backed row metadata.

This is deliberately narrower than the source’s descriptive estimates: callers that need an exact graph-input cardinality cannot substitute file sizes, optimizer statistics, or an unselected row-group total.

pub fn graph_vertex_upper_bound(&self) -> Option<u64>

Returns a footer-proven graph vertex upper bound when this source still denotes complete, unfiltered edge-file rowsets.

A graph vertex count belongs to the exact source rowset that produced it. A scan limit, reader predicate, or selected row-group subset changes that rowset, so the method returns None and does not carry the file-level value across such a transformation. Multiple whole files are safe to sum: duplicate vertices only make the result more conservative.

pub fn has_simple_directed_edge_topology( &self, source_column: &str, destination_column: &str, ) -> bool

Returns whether this source proves a simple directed edge relation for the two requested physical file columns.

A file footer cannot prove that endpoint pairs are unique across several files, so this accepts a single complete local Parquet file. Column roles must resolve directly to physical file columns, which excludes aliases, partition columns, and materialized null columns from the proof.

pub fn is_direct_device_read(&self) -> bool

Whether diagnostics indicate that direct device read is supported.

pub fn partition_column_count(&self) -> usize

Number of synthetic partition columns in the source schema.

pub fn source_column_ref(&self, column: &str) -> Option<SourceColumnRef>

Resolves a logical source column to a file, partition, or missing-null column reference.

Trait Implementations

impl Clone for LocalParquetSource

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LocalParquetSource

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for LocalParquetSource

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for LocalParquetSource