Struct ParquetStreamingSink 

pub struct ParquetStreamingSink<T: ToParquetBatch + Send> { /* private fields */ }

Parquet streaming sink that buffers items and writes in row groups.

Items are buffered until a row group is full, then written to the Parquet file. This approach provides efficient compression while supporting streaming input.

Type Parameters

• T - The type of items to write. Must implement ToParquetBatch.

Note

For custom types, implement the ToParquetBatch trait to define how your data maps to Arrow schema and batches.

Implementations

impl<T: ToParquetBatch + Send> ParquetStreamingSink<T>

pub fn new(path: PathBuf, row_group_size: usize) -> SynthResult<Self>

Creates a new Parquet streaming sink.

Arguments

• path - Path to the output Parquet file
• row_group_size - Number of rows per row group (default: 10000)

Errors

Returns an error if the file cannot be created.

pub fn with_defaults(path: PathBuf) -> SynthResult<Self>

Creates a Parquet streaming sink with default row group size (10000).

pub fn path(&self) -> &PathBuf

Returns the path to the output file.

Trait Implementations

impl<T: ToParquetBatch + Send> StreamingSink<T> for ParquetStreamingSink<T>

fn process(&mut self, event: StreamEvent<T>) -> SynthResult<()>

Processes a stream event.

fn flush(&mut self) -> SynthResult<()>

Flushes any buffered data.

fn close(self) -> SynthResult<()>

Closes the sink and releases resources.

fn items_processed(&self) -> u64

Returns the number of items processed.