Trait NetworkBoundary

Source

pub trait NetworkBoundary: ExecutionPlan {
    // Required methods
    fn with_input_stage(
        &self,
        input_stage: Stage,
    ) -> Result<Arc<dyn NetworkBoundary>>;
    fn input_stage(&self) -> &Stage;
    fn producer_head(&self, consumer_tasks: usize) -> Result<ProducerHead>;
}

Expand description

This trait represents a node that introduces the necessity of a network boundary in the plan. The distributed planner, upon stepping into one of these, will break the plan and build a stage out of it.

Required Methods§

Source

fn with_input_stage( &self, input_stage: Stage, ) -> Result<Arc<dyn NetworkBoundary>>

Called when a Stage is correctly formed. The NetworkBoundary can use this information to perform any internal transformations necessary for distributed execution.

Typically, NetworkBoundarys will use this call for transitioning from “Pending” to “ready”.

Source

fn input_stage(&self) -> &Stage

Returns the assigned input Stage, if any.

Source

fn producer_head(&self, consumer_tasks: usize) -> Result<ProducerHead>

Defines what head node should the producer stage feeding this NetworkBoundary implementation have. This information is used during planning an executing for ensuring the head of a stage has the appropriate shape for consumption. Returns an error when that shape cannot be constructed from the boundary’s partitioning.