Skip to main content

Module native_exec

Module native_exec 

Source
Expand description

DataFusion ExecutionPlan wrappers around native cuDF query plans.

Host-sink wrappers materialize native results back into Arrow streams for DataFusion. Composed wrappers can pass GPU-resident fragment results between native subplans, but the final DataFusion-facing boundary still streams host Arrow.

ExecutionPlan::execute on these wrappers performs no GPU work and claims no grant; it returns a stream whose first poll admits the attempt, runs the fragment’s whole schedule on the device’s GPU execution pool, and keeps the result on the device. Each later poll moves one bounded chunk to host memory under a DataFusion memory-pool reservation. Every failure crosses into DataFusion once, as DataFusionError::External around an adapter Error whose facts keep the native classification; a failure after native execution started ends the attempt without a CPU replay. A GPU-resident producer’s output belongs to the context of the composed root that runs it and is released with that root’s stream. The lifetime contract is in website/docs/design/admission.mdx#resident-handoff-and-release.

Structs§

AggregateStrategyExecutionOptions
Runtime threshold policy for grouped aggregate strategy selection.
AlgeonNativeExec
One native execution unit: an admitted native plan, the GPU-resident producers it reads, and its host or GPU-resident output.
AlgeonNativeExecutionConfig
Adapter-side native execution configuration.
DynamicFilterExecutionOptions
Runtime and route-discovery thresholds for native dynamic filters.
KeyedHashJoinStrategyOptions
Threshold policy used by capability admission and runtime keyed-hash join strategy selection.
LakehouseByteCache
Local byte cache for s3:// Parquet objects.
LakehouseCacheFillConfig
Service-wide resources used only by complete-object lakehouse cache fills.
PlanningPolicy
Thresholds applied during plan optimization, admission, and execution.
QueryId
Query-wide diagnostic correlation id.
RuntimeObservationConfig
Query-local recorder configuration.
RuntimeObservationLimits
Recorder limits copied into the manifest.
S3SourceStorageProfile
S3-compatible storage settings for source object-store fetches.
SealedAttemptGraphClaimCalibrationSnapshot
Immutable, typed graph claim-versus-peak rows projected by one sealed attempt.
SourceChunkPlanningOptions
Expert policy for local Parquet source chunk planning heuristics.
SourceChunking
Source chunking of native reads. Device capacity always comes from the admitted grant, so a chunking carries no device budget.
SourceSchedulerPolicy
Native source scheduling controls.
TraceId
Optional parent trace id used to join multiple query ids under one request.

Enums§

GraphClaimCalibrationExecution
Typed graph-execution classification sealed with an attempt’s calibration rows. A physical classification requires the engine to record that the graph executor started; peak qualification remains fail-closed downstream.
GraphClaimCalibrationNonPhysicalReason
Closed vocabulary for graph attempts that intentionally did not execute a graph algorithm locally.
LakehouseByteCacheFill
When a local byte-cache miss for an s3:// Parquet object is filled.
SourceChunkPlanningMode
How source chunk planning sizes local Parquet chunks. The chosen mode and the reason for it are recorded in each chunk plan.
StatisticalAggregateExecutionMode
Execution contract for floating statistical aggregates such as STDDEV_SAMP, STDDEV_POP, VAR_SAMP, and VAR_POP.

Constants§

NATIVE_ALGEON_DATAFUSION_METRIC_NAMES
Stable public DataFusion metric-name contract exposed by native cuDF execution plans.
SERVER_CAPTURE_LOSS_METRIC_NAME
DataFusion counter name for server runtime-artifact capture loss.

Functions§

below_native_admission
The plan below the admission node the native optimizer places at a query’s root, or plan itself when it has none.