Module partition

Module partition 

Source
Expand description

Arrow-based partitioning implementation for Iceberg tables

This module provides functionality to partition Arrow record batches according to Iceberg partition specifications. It includes:

  • Streaming partition implementation that processes record batches asynchronously
  • Support for different partition transforms (identity, bucket, truncate)
  • Efficient handling of distinct partition values
  • Automatic management of partition streams and channels

Functionsยง

partition_record_batch
Partitions a record batch according to the given partition fields.