Struct datafusion::physical_plan::memory::MemoryExec
source · pub struct MemoryExec { /* private fields */ }
Expand description
Execution plan for reading in-memory batches of data
Implementations§
source§impl MemoryExec
impl MemoryExec
sourcepub fn try_new(
partitions: &[Vec<RecordBatch>],
schema: Arc<Schema>,
projection: Option<Vec<usize>>,
) -> Result<MemoryExec, DataFusionError>
pub fn try_new( partitions: &[Vec<RecordBatch>], schema: Arc<Schema>, projection: Option<Vec<usize>>, ) -> Result<MemoryExec, DataFusionError>
Create a new execution plan for reading in-memory record batches
The provided schema
should not have the projection applied.
sourcepub fn with_show_sizes(self, show_sizes: bool) -> MemoryExec
pub fn with_show_sizes(self, show_sizes: bool) -> MemoryExec
set show_sizes
to determine whether to display partition sizes
pub fn partitions(&self) -> &[Vec<RecordBatch>]
pub fn projection(&self) -> &Option<Vec<usize>>
sourcepub fn with_sort_information(
self,
sort_information: Vec<Vec<PhysicalSortExpr>>,
) -> MemoryExec
pub fn with_sort_information( self, sort_information: Vec<Vec<PhysicalSortExpr>>, ) -> MemoryExec
A memory table can be ordered by multiple expressions simultaneously.
EquivalenceProperties
keeps track of expressions that describe the
global ordering of the schema. These columns are not necessarily same; e.g.
┌-------┐
| a | b |
|---|---|
| 1 | 9 |
| 2 | 8 |
| 3 | 7 |
| 5 | 5 |
└---┴---┘
where both a ASC
and b DESC
can describe the table ordering. With
EquivalenceProperties
, we can keep track of these equivalences
and treat a ASC
and b DESC
as the same ordering requirement.
pub fn original_schema(&self) -> Arc<Schema>
Trait Implementations§
source§impl Debug for MemoryExec
impl Debug for MemoryExec
source§impl DisplayAs for MemoryExec
impl DisplayAs for MemoryExec
source§impl ExecutionPlan for MemoryExec
impl ExecutionPlan for MemoryExec
source§fn as_any(&self) -> &(dyn Any + 'static)
fn as_any(&self) -> &(dyn Any + 'static)
Return a reference to Any that can be used for downcasting
source§fn statistics(&self) -> Result<Statistics, DataFusionError>
fn statistics(&self) -> Result<Statistics, DataFusionError>
We recompute the statistics dynamically from the arrow metadata as it is pretty cheap to do so
source§fn properties(&self) -> &PlanProperties
fn properties(&self) -> &PlanProperties
ExecutionPlan
, such as output
ordering(s), partitioning information etc. Read moresource§fn children(&self) -> Vec<&Arc<dyn ExecutionPlan>>
fn children(&self) -> Vec<&Arc<dyn ExecutionPlan>>
ExecutionPlan
s that act as inputs to this plan.
The returned list will be empty for leaf nodes such as scans, will contain
a single value for unary nodes, or two values for binary nodes (such as
joins).source§fn with_new_children(
self: Arc<MemoryExec>,
children: Vec<Arc<dyn ExecutionPlan>>,
) -> Result<Arc<dyn ExecutionPlan>, DataFusionError>
fn with_new_children( self: Arc<MemoryExec>, children: Vec<Arc<dyn ExecutionPlan>>, ) -> Result<Arc<dyn ExecutionPlan>, DataFusionError>
ExecutionPlan
where all existing children were replaced
by the children
, in ordersource§fn execute(
&self,
partition: usize,
_context: Arc<TaskContext>,
) -> Result<Pin<Box<dyn RecordBatchStream<Item = Result<RecordBatch, DataFusionError>> + Send>>, DataFusionError>
fn execute( &self, partition: usize, _context: Arc<TaskContext>, ) -> Result<Pin<Box<dyn RecordBatchStream<Item = Result<RecordBatch, DataFusionError>> + Send>>, DataFusionError>
source§fn static_name() -> &'static strwhere
Self: Sized,
fn static_name() -> &'static strwhere
Self: Sized,
name
but can be called without an instance.source§fn required_input_distribution(&self) -> Vec<Distribution>
fn required_input_distribution(&self) -> Vec<Distribution>
ExecutionPlan
, By default it’s [Distribution::UnspecifiedDistribution] for each child,source§fn required_input_ordering(&self) -> Vec<Option<Vec<PhysicalSortRequirement>>>
fn required_input_ordering(&self) -> Vec<Option<Vec<PhysicalSortRequirement>>>
ExecutionPlan
. Read moresource§fn maintains_input_order(&self) -> Vec<bool>
fn maintains_input_order(&self) -> Vec<bool>
false
if this ExecutionPlan
’s implementation may reorder
rows within or between partitions. Read moresource§fn benefits_from_input_partitioning(&self) -> Vec<bool>
fn benefits_from_input_partitioning(&self) -> Vec<bool>
ExecutionPlan
benefits from increased
parallelization at its input for each child. Read moresource§fn repartitioned(
&self,
_target_partitions: usize,
_config: &ConfigOptions,
) -> Result<Option<Arc<dyn ExecutionPlan>>, DataFusionError>
fn repartitioned( &self, _target_partitions: usize, _config: &ConfigOptions, ) -> Result<Option<Arc<dyn ExecutionPlan>>, DataFusionError>
ExecutionPlan
to
produce target_partitions
partitions. Read moresource§fn metrics(&self) -> Option<MetricsSet>
fn metrics(&self) -> Option<MetricsSet>
Metric
s for this
ExecutionPlan
. If no Metric
s are available, return None. Read moreAuto Trait Implementations§
impl Freeze for MemoryExec
impl !RefUnwindSafe for MemoryExec
impl Send for MemoryExec
impl Sync for MemoryExec
impl Unpin for MemoryExec
impl !UnwindSafe for MemoryExec
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<T> IntoEither for T
impl<T> IntoEither for T
source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moresource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read more