Crate ballista_executor

Expand description

§Ballista Executor Process

This crate contains the Ballista executor process.

collect: Execution plan for collecting distributed query results into a single partition. The CollectExec operator retrieves results from the cluster and returns them as a single vector of RecordBatch.
config: Command-line configuration for the executor binary. Command-line configuration parsing for the Ballista executor binary.
execution_engine: Extension point for custom query stage execution engines. Execution engine abstraction for query stage execution.
execution_loop: Pull-based task execution loop that polls the scheduler for work. Pull-based task execution loop for the executor.
executor: Core executor implementation for running distributed query tasks. Ballista executor logic
executor_process: Executor process lifecycle management and configuration. Ballista Executor Process
executor_server: gRPC server for receiving pushed tasks from the scheduler. Executor gRPC server for push-based task scheduling.
flight_service: Arrow Flight service for streaming shuffle data between executors. Implementation of the Apache Arrow Flight protocol that wraps an executor.
metrics: Metrics collection for executor runtime statistics.
shutdown: Graceful shutdown coordination for executor components. Graceful shutdown coordination for executor components.
terminate: Signal handling for process termination. Cross-platform signal handling for process termination.

as_task_status: Converts a task execution result into a TaskStatus protobuf message.
new_standalone_executor: Creates standalone executor with most values set as default.
new_standalone_executor_from_builder: Creates a standalone executor with custom configuration.
new_standalone_executor_from_state: Creates new standalone executor based on session_state provided.

ArrowFlightServerProvider: ArrowFlightServerProvider provides a function which creates a new Arrow Flight server.