Crate ballista_executor 

Ballista Executor Process

This crate contains the Ballista executor process.

Modules

collect

Execution plan for collecting distributed query results into a single partition. The CollectExec operator retrieves results from the cluster and returns them as a single vector of RecordBatch.

config

Command-line configuration for the executor binary. Command-line configuration parsing for the Ballista executor binary.

execution_engine

Extension point for custom query stage execution engines. Execution engine abstraction for query stage execution.

execution_loop

Pull-based task execution loop that polls the scheduler for work. Pull-based task execution loop for the executor.

executor

Core executor implementation for running distributed query tasks. Ballista executor logic

executor_process

Executor process lifecycle management and configuration. Ballista Executor Process

executor_server

gRPC server for receiving pushed tasks from the scheduler. Executor gRPC server for push-based task scheduling.

flight_service

Arrow Flight service for streaming shuffle data between executors. Implementation of the Apache Arrow Flight protocol that wraps an executor.

metrics

Metrics collection for executor runtime statistics.

runtime_cache

Session-scoped cache of shared executor runtime environments. Session-scoped reuse of executor runtime state.

shutdown

Graceful shutdown coordination for executor components. Graceful shutdown coordination for executor components.

terminate

Signal handling for process termination. Cross-platform signal handling for process termination.

Structs

TaskExecutionTimes

Timestamps capturing the lifecycle of a task execution.

Functions

as_task_status

Converts a task execution result into a TaskStatus protobuf message.

new_standalone_executor

Creates standalone executor with most values set as default.

new_standalone_executor_from_builder

Creates a standalone executor with custom configuration.

new_standalone_executor_from_state

Creates new standalone executor based on session_state provided.

Type Aliases

ArrowFlightServerProvider

ArrowFlightServerProvider provides a function which creates a new Arrow Flight server.