Module steps 

INLINE STEPS ON THE BATCH: select / rename / drop / filter / applyExpression applied to an Arrow RecordBatch, with the value dialect compiled through fv-value-datafusion into DataFusion physical expressions (vectorised over the columns; a sub-expression the translator cannot reproduce exactly runs as a per-row UDF inside the same expression, so semantics never change — only speed).

Compilation depends on the batch’s schema (column types decide what is exact), so a chain compiles lazily on the first batch and again whenever the input schema changes. A step that cannot be compiled against the schema at all (a column of a type the dialect has no kind for) makes the whole chain fall back to the row path — today’s behaviour, logged once.

Errors are isolated per row exactly as before: a batch that fails to evaluate (the dialect’s run-time errors, raised through the translator’s error UDF) is re-applied one row at a time, the survivors kept and the poison rows counted.

Structs

BatchSteps

A chain of inline steps compiled for one input schema.

NotCompilable

Why a chain could not be compiled for a schema (the caller falls back to rows).

Constants

META_PREFIX

Columns with this prefix are the engine’s routing metadata (offset, key): a select keeps them, and they are stripped before the rows reach an operator or an output. Every meta column starts with this: the engine strips them at the edges.

Functions

kind_type

The Arrow type a dialect kind lands in; re-exported so callers can size declared columns.