Trait deltalake::datafusion::logical_expr::logical_plan::UserDefinedLogicalNode

source ·

pub trait UserDefinedLogicalNode: Debug + Send + Sync {
    // Required methods
    fn as_any(&self) -> &(dyn Any + 'static);
    fn name(&self) -> &str;
    fn inputs(&self) -> Vec<&LogicalPlan>;
    fn schema(&self) -> &Arc<DFSchema>;
    fn expressions(&self) -> Vec<Expr>;
    fn fmt_for_explain(&self, f: &mut Formatter<'_>) -> Result<(), Error>;
    fn from_template(
        &self,
        exprs: &[Expr],
        inputs: &[LogicalPlan]
    ) -> Arc<dyn UserDefinedLogicalNode>;
    fn dyn_hash(&self, state: &mut dyn Hasher);
    fn dyn_eq(&self, other: &dyn UserDefinedLogicalNode) -> bool;

    // Provided methods
    fn prevent_predicate_push_down_columns(&self) -> HashSet<String> { ... }
    fn necessary_children_exprs(
        &self,
        _output_columns: &[usize]
    ) -> Option<Vec<Vec<usize>>> { ... }
}

Expand description

This defines the interface for LogicalPlan nodes that can be used to extend DataFusion with custom relational operators.

The UserDefinedLogicalNodeCore trait is the recommended way to implement this trait and avoids having implementing some required boiler plate code.

Required Methods§

source

fn as_any(&self) -> &(dyn Any + 'static)

Return a reference to self as Any, to support dynamic downcasting

Typically this will look like:


  // canonical boiler plate
  fn as_any(&self) -> &dyn Any {
     self
  }

source

fn name(&self) -> &str

Return the plan’s name.

source

fn inputs(&self) -> Vec<&LogicalPlan>

Return the logical plan’s inputs.

source

fn schema(&self) -> &Arc<DFSchema>

Return the output schema of this logical plan node.

source

fn expressions(&self) -> Vec<Expr>

Returns all expressions in the current logical plan node. This should not include expressions of any inputs (aka non-recursively).

These expressions are used for optimizer passes and rewrites. See LogicalPlan::expressions for more details.

source

fn fmt_for_explain(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Write a single line, human readable string to f for use in explain plan.

For example: TopK: k=10

source

fn from_template( &self, exprs: &[Expr], inputs: &[LogicalPlan] ) -> Arc<dyn UserDefinedLogicalNode>

Create a new ExtensionPlanNode with the specified children and expressions. This function is used during optimization when the plan is being rewritten and a new instance of the ExtensionPlanNode must be created.

Note that exprs and inputs are in the same order as the result of self.inputs and self.exprs.

So, `self.from_template(exprs, ..).expressions() == exprs

source

fn dyn_hash(&self, state: &mut dyn Hasher)

Update the hash state with this node requirements from Hash.

Note: consider using UserDefinedLogicalNodeCore instead of UserDefinedLogicalNode directly.

This method is required to support hashing LogicalPlans. To implement it, typically the type implementing UserDefinedLogicalNode typically implements Hash and then the following boiler plate is used:

§Example:

// User defined node that derives Hash
#[derive(Hash, Debug, PartialEq, Eq)]
struct MyNode {
  val: u64
}

// impl UserDefinedLogicalNode {
// ...
  // Boiler plate to call the derived Hash impl
  fn dyn_hash(&self, state: &mut dyn std::hash::Hasher) {
    use std::hash::Hash;
    let mut s = state;
    self.hash(&mut s);
  }
// }

Note: UserDefinedLogicalNode is not constrained by Hash directly because it must remain object safe.

source

fn dyn_eq(&self, other: &dyn UserDefinedLogicalNode) -> bool

Compare other, respecting requirements from std::cmp::Eq.

Note: consider using UserDefinedLogicalNodeCore instead of UserDefinedLogicalNode directly.

When other has an another type than self, then the values are not equal.

This method is required to support Eq on LogicalPlans. To implement it, typically the type implementing UserDefinedLogicalNode typically implements Eq and then the following boiler plate is used:

§Example:

// User defined node that derives Eq
#[derive(Hash, Debug, PartialEq, Eq)]
struct MyNode {
  val: u64
}

// impl UserDefinedLogicalNode {
// ...
  // Boiler plate to call the derived Eq impl
  fn dyn_eq(&self, other: &dyn UserDefinedLogicalNode) -> bool {
    match other.as_any().downcast_ref::<Self>() {
      Some(o) => self == o,
      None => false,
    }
  }
// }

Note: UserDefinedLogicalNode is not constrained by Eq directly because it must remain object safe.

Provided Methods§

source

fn prevent_predicate_push_down_columns(&self) -> HashSet<String>

A list of output columns (e.g. the names of columns in self.schema()) for which predicates can not be pushed below this node without changing the output.

By default, this returns all columns and thus prevents any predicates from being pushed below this node.

source

fn necessary_children_exprs( &self, _output_columns: &[usize] ) -> Option<Vec<Vec<usize>>>

Returns the necessary input columns for this node required to compute the columns in the output schema

This is used for projection push-down when DataFusion has determined that only a subset of the output columns of this node are needed by its parents. This API is used to tell DataFusion which, if any, of the input columns are no longer needed.

Return None, the default, if this information can not be determined. Returns Some(_) with the column indices for each child of this node that are needed to compute output_columns