Crate krabka_compression 

Kafka wire-protocol compression codecs.

Kafka uses four codecs on the wire: gzip, snappy, lz4, and zstd. Each codec has specific framing conventions.

• gzip: standard RFC-1952 gzip through flate2, which uses the pure-Rust miniz_oxide backend.
• snappy: xerial-snappy framing over snap raw blocks. Kafka does not use the standard Google Snappy stream format. It uses the xerial framing: an 8-byte magic header, two 4-byte version fields, then a sequence of u32-BE length-prefixed raw snappy chunks.
• lz4: LZ4 frame format with magic 0x04 22 4D 18, independent blocks, and 64 KiB block size. These are the defaults of KafkaLZ4BlockOutputStream.
• zstd: plain zstd at compression level 3, which is Kafka’s default.

Each codec is behind a Cargo feature: gzip, snappy, lz4, and zstd. All of them are enabled by default. If you disable a feature, the API stays the same, but the call returns Err(CompressionError::FeatureDisabled) at runtime.

Compress and decompress a record payload

use krabka_compression::{CompressionType, compress, decompress};
use krabka_units::kibibytes;

let compressed = compress(CompressionType::Lz4, b"order-created")?;
let plain = decompress(CompressionType::Lz4, &compressed, kibibytes(1))?;
assert_eq!(plain.as_ref(), b"order-created");

Structs

RecordDecompressionPolicy

Validated limits for decompressing untrusted Kafka record payloads.

Enums

CompressionError

Errors that can occur during compression or decompression.

CompressionType

Codec identifier matching the lowest three bits of Kafka’s record-batch attribute byte.

Constants

RECORD_DECOMPRESSION_HARD_MAX_OUTPUT

Fixed security ceiling for a decompressed record payload.

RECORD_DECOMPRESSION_HARD_MAX_RATIO

Fixed security ceiling for record decompression expansion.

Functions

compress

Compress data using the codec identified by ct.

compress_with_level

Compress data using the codec identified by ct at compression level (KIP-390).

decompress

Decompress data using the codec identified by ct. See compress.