Skip to main content

Module query_planner

Module query_planner 

Source
Expand description

FFI support for QueryPlanner.

A typical deployment has three libraries. Library A (for example, datafusion-python) owns the Session and codec registry. Library B owns a custom table provider and its extension nodes. Library C (for example, Ballista or datafusion-distributed) owns the query planner. A serializes a logical plan and invokes C, while FFI_SessionRef lets C call session services in A. C deserializes the logical plan, creates a physical plan, serializes that result, and returns it for A to deserialize. The logical and physical extension codecs preserve nodes supplied by B.

The physical result is serialized instead of returned as an crate::execution_plan::FFI_ExecutionPlan. An FFI execution-plan handle is a foreign trait-object proxy, so even a built-in plan created in C cannot be downcast to its concrete type in A. Serialization reconstructs known plan nodes with A’s local Rust type identities, allowing A’s optimizers and other consumers to downcast them. Extension codecs control how custom nodes are reconstructed.

A node returned by B while C is planning is still foreign to C unless a codec boundary reconstructs it in C. The query-planner boundary guarantees that C-local serializable nodes, and extension nodes understood by the configured codecs, are reconstructed for A when the completed plan returns.

§Delegating back to library A

C commonly wants A’s built-in planning as a starting point, then rewrites the result. A must export its planner before installing C’s planner on the session, and C must retain that handle: after the swap, Session::query_planner reports C’s own planner, and Session::create_physical_plan dispatches to it, so either one is a self-call. Delegating to the retained handle is safe, because DataFusion’s built-in physical planner never re-dispatches through Session.

Retain the planner rather than the session. FFI_QueryPlanner owns a reference-counted planner, so it outlives A’s original session, whereas FFI_SessionRef borrows its session with the lifetime erased.

Structs§

FFI_QueryPlanner
An ABI-stable handle to a QueryPlanner owned by another library.
ForeignQueryPlanner
Consumer-side QueryPlanner adapter for an FFI_QueryPlanner.