Enum ConvertSource 

pub enum ConvertSource {
    Single(SingleSource),
    Multi(MultiSource),
}

A resolved conversion input: one parquet file/object, or an ordered set of local parquet partitions read as one logical dataset.

Variants

Single(SingleSource)

A single parquet file or remote object (the historical input shape).

Multi(MultiSource)

An ordered, validated set of local parquet partitions.

Implementations

impl ConvertSource

pub fn single(source: InputSource) -> Self

Wrap an already-constructed InputSource (single file/object).

pub fn resolve(input: &str) -> Result<Self, InputError>

Resolve a CLI-style input string:

• existing local file → Single (byte-identical behavior to today);
• existing local directory → recursive .parquet collection (sorted; _/.-prefixed basenames such as _SUCCESS skipped);
• string containing glob metacharacters (*?[) → glob expansion, filtered to .parquet files, sorted, deduplicated;
• remote single-object URL (no trailing slash — including extension-less presigned/API URLs) → Single (unchanged);
• s3:// / gs:// prefix (path ending /) → native object listing: .parquet keys sorted by key, _SUCCESS/zero-byte/ hidden (./_) names skipped, one store instance shared by all parts; requires the remote feature (without it, the standard InputError::RemoteDisabled as before);
• http(s):// prefix → InputError::RemotePrefixUnsupported (no generic listing API; the error points at --files-from);
• a single resolved partition collapses to Single;
• an empty directory/glob result → InputError::NoParquetInputs.

pub fn from_manifest(manifest: &Path) -> Result<Self, InputError>

Build a source from a --files-from manifest: one local path or remote URL per line, #-prefixed comment lines and blank lines skipped, entries trimmed. Line order is preserved VERBATIM — never sorted — because the converter’s row-order invariant keys winner tables by global row offset; reordering the manifest reorders the dataset. Each line is resolved as a SINGLE file/object (no directory, glob, or prefix expansion); mixing local and remote entries is allowed (compatibility is validated as usual).

pub fn from_input_list<S: AsRef<str>>(inputs: &[S]) -> Result<Self, InputError>

Build a source from an explicit ordered list of inputs (local paths or URLs) — the Python list[str] input shape. Order is preserved verbatim; each entry is a single file/object (no expansion).

pub fn resolve_path(path: &Path) -> Result<Self, InputError>

ConvertSource::resolve for Path inputs (non-UTF-8 paths fall back to a single local source, as InputSource::from_path does).

pub fn parts(&self) -> &[InputSource]

The underlying parts, in read order (a single source is one part).

pub fn is_remote(&self) -> bool

Whether any part is remote.

pub fn set_spill_dir(&self, dir: Option<&Path>)

Place the remote-input disk spill in dir for every part (#272). No-op for local parts, which never spill.

pub fn display_name(&self) -> String

Human-readable input name: the path/URL for a single source, "<root> (N partitions)" for a multi source.

pub fn schema(&self) -> Result<SchemaRef, InputError>

The Arrow schema of the dataset. For a multi source this is the validated union schema (nullability OR-ed across parts). After Self::restrict_columns, only the kept columns, in file order.

pub fn file_schema(&self) -> Result<SchemaRef, InputError>

The unprojected schema: every column the files carry, whether or not Self::restrict_columns has narrowed what reads return.

pub fn restrict_columns(&self, keep: Vec<usize>) -> Result<(), InputError>

Restrict every later read — and Self::schema — to these root columns of the file schema, sorted ascending (#386). Applied once, before anything derives column indices from the schema, so every downstream index is already relative to the projected layout and the excluded columns are never decoded (a remote input still stages every chunk of a row group; only the decode is skipped). A second call is a programming error.

pub fn column_projection(&self) -> Option<&[usize]>

The root columns reads are restricted to, if any (see Self::restrict_columns).

pub fn key_value_metadata(&self) -> Result<Option<Vec<KeyValue>>, InputError>

Parquet key-value metadata of partition 0 (the geo metadata used for CRS detection; construction validated all parts agree).

pub fn num_row_groups_total(&self) -> Result<usize, InputError>

Total number of row groups across all parts.

pub fn select_row_groups( &self, bbox_units: &[f64; 4], ) -> Result<RowGroupSelection, InputError>

Per-part bbox row-group selection (#102): applies the single-file covering-statistics pruning to each part independently. bbox_units is [xmin, ymin, xmax, ymax] in the file CRS units.

pub fn select_row_groups_matching( &self, filter: &BoundFilter, ) -> Result<RowGroupSelection, InputError>

Per-part attribute-filter row-group selection (#315): applies the bound filter’s column-statistics pushdown (crate::overview::filter::BoundFilter::select_row_groups) to each part independently. Conservative — groups without usable statistics are kept.

pub fn selected_input_bytes( &self, selection: Option<&RowGroupSelection>, ) -> Result<u64, InputError>

Total compressed bytes of the selected row groups (None = every row group of every part) — the projected disk-spill size the #272 free-space preflight consults. Per-part sums come from the shared single-file helper crate::input::selected_compressed_bytes.

pub fn fetch_stats(&self) -> Option<FetchStats>

Fetch counters summed over remote parts (None when no part is remote). object_size is the summed size of the remote objects.

pub fn stage_selected( &self, selection: Option<&RowGroupSelection>, ) -> Result<(), InputError>

Stage every part’s selected row groups to local disk up front (pass 0, #286/#287) so the streaming passes read from disk, not the network.

Each remote part coalesces its selected row groups into one parallel range request per row group (a row group is a contiguous byte span); local parts are no-ops. Parts are staged in order so resident memory stays bounded by one part’s in-flight spans. selection (None = all row groups) is the per-part bbox pruning the passes will honor, so pruned groups are never fetched and total network traffic stays ≈1× the input (#219). Errors are the caller’s to handle; the streaming pipeline treats staging as best-effort (a network error here is one the passes would hit anyway).

pub fn open_stream( &self, plan: &ReadPlan<'_>, ) -> Result<SourceStream<'_>, InputError>

Open a sequential batch stream over all parts (see SourceStream).

Trait Implementations

impl Debug for ConvertSource

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.