Module compression 

Available on crate feature compression only.

Transparent gzip / zstd compression wrappers for file-shaped connectors.

Behind the compression feature. Exposes:

• CompressionConfig — user-facing enum (None, Gzip, Zstd, Auto).
• Compression — internal post-resolution enum (no Auto).
• wrap_async_reader / wrap_async_writer — for stream_pages and async sinks.
• wrap_sync_reader / wrap_sync_writer — for spawn_blocking paths.
• compress_buf — one-shot in-memory compression for S3/GCS sink uploads.
• warn_mismatch — log-once helper when explicit codec disagrees with the filename extension.

Enums

Compression

Internal post-resolution codec. No Auto variant — call CompressionConfig::resolve.

CompressionConfig

User-facing compression config. Defaults to Auto.

SyncCompressWriter

A sync compressing writer that retains the concrete encoder so its finalisation error can be captured, unlike the Box<dyn Write> returned by wrap_sync_writer (#78/#41).

Functions

compress_buf

One-shot in-memory compression. Used by S3 and GCS sinks that build a full Vec<u8> body before upload.

detect_from_path

Codec detection by filename suffix. Case-insensitive. Looks at the final extension only: foo.csv.gz → Gzip, foo.gz.zst → Zstd.

sync_compress_writer

Wrap a sync writer in a SyncCompressWriter for the given codec. Prefer this over wrap_sync_writer when you need to detect a finalisation error: call SyncCompressWriter::finish before dropping.

warn_mismatch

Log a one-shot warning when the explicit codec disagrees with the filename’s detected codec. Deduplicates per (path, declared) pair across the whole process so a million-object scan does not flood logs.

wrap_async_reader

Wrap an async buffered reader with a streaming decoder. None passes through.

wrap_async_writer

Wrap an async writer with a streaming encoder. None passes through. Callers must call tokio::io::AsyncWriteExt::shutdown on the returned writer to flush the trailer before the underlying writer is dropped.

wrap_sync_reader

Wrap a sync reader with a streaming decoder. None passes through.

wrap_sync_writer

Wrap a sync writer with a streaming encoder. None passes through.