Enum InputSource 

pub enum InputSource {
    Local(PathBuf),
}

A conversion input: a local parquet file or a remote parquet object.

Clone is cheap: the local variant clones a path, the remote variant clones Arc handles — clones share the fetch counters, chunk cache, disk spill, and cached footer of the original.

Variants

Local(PathBuf)

A local filesystem path (the historical behavior).

Implementations

impl InputSource

pub fn from_path(path: &Path) -> Result<Self, InputError>

Classify a CLI-style input path. Anything shaped like scheme://... is treated as a URL (file:// maps back to a local path); everything else is a local path.

pub fn from_str_input(input: &str) -> Result<Self, InputError>

InputSource::from_path for string inputs.

pub fn is_remote(&self) -> bool

Whether this source is remote.

pub fn display_name(&self) -> String

Human-readable name of the input (path or URL).

pub fn open( &self, ) -> Result<ParquetRecordBatchReaderBuilder<InputReader>, InputError>

Open a parquet reader builder over this input.

Local: opens the file and parses the footer (cheap, OS page cache). Remote: reuses a cached parsed footer after the first open, so the multi-pass streaming pipeline pays the footer fetch only once.

pub fn fetch_stats(&self) -> Option<FetchStats>

Fetch counters for a remote source (None for local inputs).

pub fn fetched_ranges(&self) -> Option<Vec<Range<u64>>>

The byte ranges fetched so far from a remote source (None for local inputs). Ordered by request time; used by tests to prove that bbox-pruned row groups are never downloaded.

pub fn set_spill_dir(&self, dir: Option<&Path>)

Place the remote-input disk spill (#219) in dir instead of the process temp dir ($TMPDIR) — issue #272. No-op for local inputs, which never spill. Call before reading: chunks already spilled stay in the previously created file; only the spill-file creation (lazy, on the first spilled chunk) honors the directory.

pub fn release_read_cache(&self)

Release the in-memory read cache (no-op for local files). For a remote source this clears the L1 chunk cache but KEEPS the disk spill and the cached footer, so a later touch of the same chunk is served from local disk, not the network. Called by multi-partition streams on part transitions to bound resident memory to one part’s working set.

pub fn stage_row_groups( &self, selected: Option<&[usize]>, ) -> Result<(), InputError>

Stage the selected row groups to the local disk spill up front (pass 0, #286/#287), coalescing each row group into one parallel range request so the later passes read entirely from disk. No-op for local inputs (the OS page cache already serves re-reads). selected (None = all) must match the row groups the passes will read, so pruned groups are never fetched and total traffic stays ≈1× (#219).

Trait Implementations

impl Clone for InputSource

fn clone(&self) -> InputSource

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for InputSource

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.