pub struct ExecutionPlan {Show 13 fields
pub field_map: FxHashMap<String, String>,
pub drop_fields: FxHashSet<String>,
pub field_types: FxHashMap<String, FieldType>,
pub dictionary_columns: FxHashSet<String>,
pub filter: Option<FilterPredicate>,
pub schema_order: Vec<String>,
pub auto_dict: bool,
pub dict_threshold: Option<f64>,
pub dict_max_size: Option<usize>,
pub strict_types: bool,
pub max_split_chunks: Option<usize>,
pub min_chunk_bytes: Option<usize>,
pub observer: Option<Arc<dyn RowObserver>>,
}Expand description
A compiled execution plan that controls field renaming, dropping, type assignment, dictionary encoding, row filtering, and column ordering. Default (empty) is a no-op.
Fields§
§field_map: FxHashMap<String, String>Map from raw field name to output column name.
drop_fields: FxHashSet<String>Set of raw field names to drop entirely.
field_types: FxHashMap<String, FieldType>Explicit type overrides per output column name.
dictionary_columns: FxHashSet<String>Set of output column names to dict-encode.
filter: Option<FilterPredicate>Optional row filter predicate.
schema_order: Vec<String>Desired output column order (names in order). Columns not listed here appear after all listed columns in first-appearance order. If empty, first-appearance order is used.
auto_dict: boolWhen true, string columns with low cardinality are automatically upgraded to dictionary encoding during parse.
dict_threshold: Option<f64>Auto-dict tuning: maximum fraction of rows allowed as distinct values
for an upgrade. Defaults to 0.05 when None.
dict_max_size: Option<usize>Auto-dict tuning: maximum dictionary size (distinct values) allowed for
an upgrade. Defaults to 256 when None.
strict_types: boolWhen true, a non-null value that fails to parse into its declared
field_types type aborts the parse with an error instead of silently
becoming null. Genuine nulls (missing fields) stay null regardless.
max_split_chunks: Option<usize>Cap on the number of bounded-streaming batches/chunks. Defaults to
MAX_SPLIT_CHUNKS (100,000) when None.
min_chunk_bytes: Option<usize>Opt-in chunk-size floor for split planning. None uses the engine
default ([crate::MIN_CHUNK_BYTES], 2 MiB). Lower it for many small
cores (more, smaller chunks); raise it for fewer, larger chunks.
observer: Option<Arc<dyn RowObserver>>Optional row observer; hooks fire from parse threads during the read.
Implementations§
Source§impl ExecutionPlan
impl ExecutionPlan
pub fn new() -> Self
Sourcepub fn rename(self, raw: impl Into<String>, output: impl Into<String>) -> Self
pub fn rename(self, raw: impl Into<String>, output: impl Into<String>) -> Self
Rename a raw field to an output column name.
Sourcepub fn type_as(self, field: impl Into<String>, field_type: FieldType) -> Self
pub fn type_as(self, field: impl Into<String>, field_type: FieldType) -> Self
Set the storage type for an output column.
Sourcepub fn dictionary(self, field: impl Into<String>) -> Self
pub fn dictionary(self, field: impl Into<String>) -> Self
Dict-encode an output column.
Sourcepub fn filter_eq(
self,
field: impl Into<String>,
value: impl Into<String>,
) -> Self
pub fn filter_eq( self, field: impl Into<String>, value: impl Into<String>, ) -> Self
Keep only rows where field == value (string comparison, per-row).
Sourcepub fn filter_ne(
self,
field: impl Into<String>,
value: impl Into<String>,
) -> Self
pub fn filter_ne( self, field: impl Into<String>, value: impl Into<String>, ) -> Self
Keep only rows where field != value (string comparison, per-row).
Sourcepub fn filter_compare(
self,
field_a: impl Into<String>,
op: CompareOp,
field_b: impl Into<String>,
) -> Self
pub fn filter_compare( self, field_a: impl Into<String>, op: CompareOp, field_b: impl Into<String>, ) -> Self
Keep only rows where field_a op field_b (native-typed, per-row).
Sourcepub fn schema_order<I, S>(self, order: I) -> Self
pub fn schema_order<I, S>(self, order: I) -> Self
Declare the output schema as a projection + order: the output
contains exactly the listed columns, in the listed order. Fields in
the data but not listed are skipped during parsing (never
materialized); listed columns absent from the data come out
null-filled. field_map renames apply before matching.
Sourcepub fn with_auto_dict(self, yes: bool) -> Self
pub fn with_auto_dict(self, yes: bool) -> Self
Enable or disable automatic dictionary upgrade for low-cardinality string columns.
Sourcepub fn with_dict_threshold(self, threshold: f64) -> Self
pub fn with_dict_threshold(self, threshold: f64) -> Self
Override the auto-dict distinct-ratio threshold (default 0.05).
Sourcepub fn with_dict_max_size(self, max_size: usize) -> Self
pub fn with_dict_max_size(self, max_size: usize) -> Self
Override the auto-dict maximum dictionary size (default 256).
Sourcepub fn with_strict_types(self, yes: bool) -> Self
pub fn with_strict_types(self, yes: bool) -> Self
Enable strict typing: a non-null value that cannot be parsed into its
field_types-declared type becomes an error at finish() instead of
a silent null.
Sourcepub fn with_max_split_chunks(self, cap: usize) -> Self
pub fn with_max_split_chunks(self, cap: usize) -> Self
Override the bounded-streaming split cap (default 100,000 chunks).
Sourcepub fn with_min_chunk_bytes(self, bytes: usize) -> Self
pub fn with_min_chunk_bytes(self, bytes: usize) -> Self
Override the split-planning chunk-size floor (default 2 MiB).
Sourcepub fn with_observer(self, observer: Arc<dyn RowObserver>) -> Self
pub fn with_observer(self, observer: Arc<dyn RowObserver>) -> Self
Attach a row observer; hooks fire from parse threads during the read.
Sourcepub fn column_type(&self, name: &str) -> FieldType
pub fn column_type(&self, name: &str) -> FieldType
Determine the storage type for an output column name.
Sourcepub fn resolve_field<'a>(&'a self, raw: &'a str) -> Option<&'a str>
pub fn resolve_field<'a>(&'a self, raw: &'a str) -> Option<&'a str>
Resolve a raw field name to its output column name.
Returns None if the field should be dropped.
Application order: rename first, then drop, then schema projection, matching left-to-right pipeline semantics.
When schema_order is non-empty it acts as a projection: only the
listed columns are materialized, so any other resolved name returns
None here (which makes ColumnarSink::wants/resolve reject the
field and lets adapters skip extracting it). Fields referenced by the
row filter are kept even when absent from schema_order so predicates
still evaluate correctly; they are projected out of the final batch.
Trait Implementations§
Source§impl Clone for ExecutionPlan
impl Clone for ExecutionPlan
Source§impl Debug for ExecutionPlan
impl Debug for ExecutionPlan
Auto Trait Implementations§
impl !RefUnwindSafe for ExecutionPlan
impl !UnwindSafe for ExecutionPlan
impl Freeze for ExecutionPlan
impl Send for ExecutionPlan
impl Sync for ExecutionPlan
impl Unpin for ExecutionPlan
impl UnsafeUnpin for ExecutionPlan
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more