Struct HanaBatchProcessor

Source

pub struct HanaBatchProcessor { /* private fields */ }

Expand description

Processor that converts HANA rows into Arrow RecordBatches.

Buffers rows until batch_size is reached, then emits a RecordBatch. Implements the BatchProcessor trait with GAT support.

§Example

use hdbconnect_arrow::conversion::HanaBatchProcessor;
use hdbconnect_arrow::traits::streaming::BatchConfig;

let schema = /* Arrow schema */;
let config = BatchConfig::with_batch_size(10000);
let mut processor = HanaBatchProcessor::new(Arc::new(schema), config);

for row in result_set {
    if let Some(batch) = processor.process_row(row)? {
        // Process batch
    }
}

// Don't forget to flush remaining rows
if let Some(batch) = processor.flush()? {
    // Process final batch
}

Implementations§

Source §

impl HanaBatchProcessor

Source

pub fn new(schema: SchemaRef, config: BatchConfig) -> Self

Create a new batch processor.

§Arguments

schema - Arrow schema for the batches
config - Batch processing configuration

Source

pub fn with_defaults(schema: SchemaRef) -> Self

Create with default configuration.

Source

pub fn process_row(&mut self, row: &Row) -> Result<Option<RecordBatch>>

Process a single row.

Returns Ok(Some(batch)) when a batch is ready, Ok(None) when more rows are needed to fill a batch.

§Errors

Returns error if value conversion fails or schema mismatches.

Source

pub fn process_row_generic<R: RowLike>( &mut self, row: &R, ) -> Result<Option<RecordBatch>>

Process a single row using the generic RowLike trait.

This method enables unit testing with MockRow instead of requiring a HANA connection.

Returns Ok(Some(batch)) when a batch is ready, Ok(None) when more rows are needed to fill a batch.

§Errors

Returns error if value conversion fails or schema mismatches.

§Example

use hdbconnect_arrow::traits::row::{MockRow, MockRowBuilder};

let row = MockRowBuilder::new().int(42).string("test").build();
let result = processor.process_row_generic(&row)?;

Source