Trait MapPandasIterUdf 

pub trait MapPandasIterUdf:
    Send
    + Sync
    + Debug {
    // Required methods
    fn name(&self) -> &str;
    fn input_schema(&self) -> &Schema;
    fn output_schema(&self) -> &Schema;
    fn map_batches(
        &self,
        batches: &[RecordBatch],
    ) -> Result<Vec<RecordBatch>, UdfError>;
}

A stateful iterator-over-batches map function.

Receives batches from one partition one at a time; may return multiple output batches per input batch. The Python callable receives a pandas DataFrame iterator and must yield pandas DataFrames.

This mirrors PySpark’s mapInPandas / Flink Python DataStream map.

Required Methods

fn name(&self) -> &str

Unique name used to look up this UDF in a UdfRegistry.

fn input_schema(&self) -> &Schema

Schema of the input batches.

fn output_schema(&self) -> &Schema

Schema of the output batches.

fn map_batches( &self, batches: &[RecordBatch], ) -> Result<Vec<RecordBatch>, UdfError>

Process the provided batches and return all output batches.

Implementations may buffer or emit eagerly.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors