Expand description
Record-batch compression codecs.
The Compression enum is always available so crate::record::RecordBatch
can read the compression type from attributes regardless of which codec
features are enabled. Actual compress/decompress operations require the
corresponding Cargo feature:
| Feature | Codec | Backend | Pure Rust | HC mode |
|---|---|---|---|---|
gzip | Gzip | flate2 | yes | n/a |
snappy | Snappy | snap | yes | n/a |
lz4 | Lz4 | lz4_flex | yes | no |
lz4-hc | Lz4 | lz4 (C-FFI) | no | yes |
zstd | Zstd | zstd (C-FFI) | no | n/a |
compression is a meta-feature that enables gzip + snappy + lz4 + zstd
(pure-Rust LZ4). Opt into lz4-hc explicitly if you need
High-Compression mode — see the LZ4 section below.
§LZ4 backend selection
lz4—lz4_flexblock API behind a Kafka-compatible custom frame. Fast mode only; thelevelargument is ignored. Pure Rust, no C toolchain at build time, cross-compile clean.lz4-hc—lz4crate (FFI to liblz4) behind the same frame format. Levels 3..=12 use HC mode; levels 0..=2 fall back to fast mode. Requires a C compiler at build time.
Both features are independent — enabling lz4-hc alone is sufficient
(the C lib handles fast mode too). When both are enabled, lz4-hc
wins at runtime; lz4_flex is linked but unused. Most users should
pick one.
For high compression ratio in Kafka workloads, prefer zstd over
lz4-hc — modern Kafka clients (Java, librdkafka) default to fast LZ4
when LZ4 is selected, and switch to zstd when ratio matters.
Re-exports§
pub use self::error::CompressionError;pub use self::error::CompressionErrorKind;
Modules§
- error
- Error types for
crate::compression. - gzip
- Gzip codec (
flate2Rust backend). - lz4
- Kafka-compatible LZ4 frame codec.
- snappy
- Snappy codec (
snapRust backend). Level is accepted but ignored. - zstd
- Zstandard codec (
zstdcrate).
Enums§
- Compression
- Kafka record-batch compression codec, encoded in bits 0–2 of the batch
attributesfield.
Constants§
- MAX_
DECOMPRESSED_ LEN - Hard ceiling on the decompressed size of a single payload.
Type Aliases§
- Result
- Result alias for compression operations.