Struct ContainerDecoder 

pub struct ContainerDecoder { /* private fields */ }

Decodes a sequence of Avro or ORC files against one shape.

Implementations

impl ContainerDecoder

pub fn new( format: FileFormat, opts: &FormatOptions, ) -> Result<Self, FaucetError>

A decoder for format, which must be FileFormat::Avro or FileFormat::Orc in a build with the matching feature.

pub fn format(&self) -> FileFormat

The format this decoder reads.

pub fn decode_all( &mut self, name: &str, input: FileInput, ) -> Result<Vec<Value>, FaucetError>

Decode one whole file into its records.

pub fn decode_batches( &mut self, name: &str, input: FileInput, batch_size: usize, ) -> Result<(SchemaRef, Vec<RecordBatch>), FaucetError>

Available on crate feature arrow only.

Decode one whole file into Arrow batches of at most batch_size rows.

pub fn records( &mut self, name: &str, input: FileInput, chunk: usize, f: &mut dyn FnMut(Vec<Value>) -> Result<(), FaucetError>, ) -> Result<(), FaucetError>

Decode one file’s records in chunks of chunk (0 = the whole file as one chunk), calling f per chunk.

pub fn batches( &mut self, name: &str, input: FileInput, batch_size: usize, f: &mut dyn FnMut(RecordBatch) -> Result<(), FaucetError>, ) -> Result<SchemaRef, FaucetError>

Available on crate feature arrow only.

Decode one file as Arrow batches of at most batch_size rows (0 = the reader’s natural size), calling f per batch. Returns the file’s Arrow schema, identical for every file this decoder accepts.

Trait Implementations

impl Debug for ContainerDecoder

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.