Skip to main content

normalize_to_current

Function normalize_to_current 

Source
pub fn normalize_to_current(
    batch: &RecordBatch,
    from_version: &str,
) -> Result<RecordBatch, ArrowError>
Expand description

Normalize a RecordBatch from an older schema version to the current version.

This is the read-path migration: when loading Parquet files written with older schemas, the normalizer adds missing columns with default values.

Supported migrations:

  • v1.0.0 → v1.1.0: adds null source_chunk_id column at index 9