Skip to main content

BatchAdapterFactory

Struct BatchAdapterFactory 

Source
pub struct BatchAdapterFactory { /* private fields */ }
Expand description

Factory for creating BatchAdapter instances to adapt record batches to a target schema.

This binds a target schema and allows creating adapters for different source schemas. It handles:

  • Column reordering: Columns are reordered to match the target schema
  • Type casting: Automatic type conversion (e.g., Int32 to Int64)
  • Missing columns: Nullable columns missing from source are filled with nulls
  • Struct field adaptation: Nested struct fields are recursively adapted

§Examples

use arrow::array::{Int32Array, Int64Array, StringArray, RecordBatch};
use arrow::datatypes::{DataType, Field, Schema};
use datafusion_physical_expr_adapter::BatchAdapterFactory;
use std::sync::Arc;

// Target schema has different column order and types
let target_schema = Arc::new(Schema::new(vec![
    Field::new("name", DataType::Utf8, true),
    Field::new("id", DataType::Int64, false),    // Int64 in target
    Field::new("score", DataType::Float64, true), // Missing from source
]));

// Source schema has different column order and Int32 for id
let source_schema = Arc::new(Schema::new(vec![
    Field::new("id", DataType::Int32, false),    // Int32 in source
    Field::new("name", DataType::Utf8, true),
    // Note: 'score' column is missing from source
]));

// Create factory with target schema
let factory = BatchAdapterFactory::new(Arc::clone(&target_schema));

// Create adapter for this specific source schema
let adapter = factory.make_adapter(Arc::clone(&source_schema)).unwrap();

// Create a source batch
let source_batch = RecordBatch::try_new(
    source_schema,
    vec![
        Arc::new(Int32Array::from(vec![1, 2, 3])),
        Arc::new(StringArray::from(vec!["Alice", "Bob", "Carol"])),
    ],
).unwrap();

// Adapt the batch to match target schema
let adapted = adapter.adapt_batch(&source_batch).unwrap();

assert_eq!(adapted.num_columns(), 3);
assert_eq!(adapted.column(0).data_type(), &DataType::Utf8);   // name
assert_eq!(adapted.column(1).data_type(), &DataType::Int64);  // id (cast from Int32)
assert_eq!(adapted.column(2).data_type(), &DataType::Float64); // score (filled with nulls)

Implementations§

Source§

impl BatchAdapterFactory

Source

pub fn new(target_schema: SchemaRef) -> Self

Create a new BatchAdapterFactory with the given target schema.

Source

pub fn with_adapter_factory( self, factory: Arc<dyn PhysicalExprAdapterFactory>, ) -> Self

Set a custom PhysicalExprAdapterFactory to use when adapting expressions.

Use this to customize behavior when adapting batches, e.g. to fill in missing values with defaults instead of nulls.

See PhysicalExprAdapter for more details.

Source

pub fn make_adapter(&self, source_schema: SchemaRef) -> Result<BatchAdapter>

Create a new BatchAdapter for the given source schema.

Batches fed into this BatchAdapter must conform to the source schema, no validation is performed at runtime to minimize overheads.

Trait Implementations§

Source§

impl Debug for BatchAdapterFactory

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> ErasedDestructor for T
where T: 'static,