crabka-compression

Kafka wire-protocol compression codecs for Rust. Implements the four codecs Apache Kafka uses on the wire — gzip, snappy, lz4, zstd — with byte-level wire compatibility verified against the JVM kafka-clients implementation.

Usage example

use crabka_compression::{compress, decompress, CompressionType};

let bytes = compress(CompressionType::Snappy, b"hello kafka").unwrap();
let back  = decompress(CompressionType::Snappy, &bytes).unwrap();
assert_eq!(back.as_ref(), b"hello kafka");

Features

Default features enable all four codecs. Disable individually:

crabka-compression = { version = "0.3.2", default-features = false, features = ["gzip", "zstd"] }

Calling a codec whose feature is off returns CompressionError::FeatureDisabled.

Kafka-specific framing

Snappy uses xerial-snappy framing (Kafka does not use Google's official Snappy stream format).
LZ4 uses the LZ4 frame format with independent blocks, 64 KiB block size, no checksums.
Gzip is plain RFC-1952.
Zstd is plain zstd frame at level 3.

MSRV

Rust 1.95.0.

License