Crate transferred_files 

Local filesystem source + destination, with pluggable file-format codecs (Parquet now; Csv/Avro later). Built on the arrow-rs parquet crate.

Structs

FilesDestination

Local files destination. Writes part-NNNNN.{extension} files to a directory, or one {dir}.{extension} when single_file. Write is atomic via tmp dir + rename. Written paths land in RunReport.written_objects.

FilesSource

Local file source. One or many files, decoded by the supplied FormatRead.

Parquet

Parquet file format. Carries encoder knobs; decoding needs none.

Enums

Compression

Compression codec for Parquet column chunks. Default = Zstd.

GlobOrPaths

How the source enumerates files: glob or single path, or list of paths.

Traits

FileReader

A readable file handle trait marker for random-access bytes.

FileWriter

A writable file handle trait.

FormatRead

Decodes a file’s bytes into Arrow batches.

FormatWrite

Encodes Arrow batches into a file’s bytes.