mzstd

Zstandard (RFC 8878) compression and decompression in safe Rust.

• Byte-identical encoder. For the same input and parameters, single-threaded compression writes exactly the bytes of zstd 1.5.7: one-shot calls match ZSTD_compress2, and sessions match ZSTD_compressStream2 driven with the same chunks, output capacities and flushes. This holds for levels -131072 to 22, the explicit compression parameters, long distance matching, checksums, content sizes, pledged sizes and size hints.
• Independent decoder written from the format specification. It decodes every frame zstd writes apart from dictionary frames, with bounded window, input and output limits.
• Parallel compression with tasks you run on your own executor. Frames are valid Zstandard frames and are identical for any task count, but they are not byte-identical to zstd's multithreaded mode.
• No unsafe, no dependencies on zstd. It runs with no_std + alloc, and there is a C ABI in mzstd-ffi.

[dependencies]
mzstd = "0.1"

Usage

use mzstd::{Compressor, Decompressor, EncoderConfig, Level};

let mut encoder = Compressor::new(EncoderConfig::default().with_level(Level::try_from(19)?))?;
let compressed = encoder.compress(b"hello, hello, hello")?;

let restored = Decompressor::default().decompress(&compressed)?;
assert_eq!(restored, b"hello, hello, hello");
# Ok::<(), Box<dyn std::error::Error>>(())

Explicit parameters override the level just as ZSTD_CCtx_setParameter does:

use mzstd::{CompressionParams, Compressor, EncoderConfig, LdmParams, Level, LongDistanceMatching, Strategy};

let config = EncoderConfig::default()
    .with_level(Level::try_from(9)?)
    .with_checksum(true)
    .with_params(CompressionParams::default().with_window_log(24).with_strategy(Strategy::Lazy2))
    .with_long_distance_matching(LongDistanceMatching::Enabled(LdmParams::default()));
let mut encoder = Compressor::new(config)?;
# Ok::<(), Box<dyn std::error::Error>>(())

Features

Performance

Speed was not a goal of this first version: both codecs are scalar ports without SIMD. As a reference point, here is cargo bench --bench compress and --bench decompress on an Apple M5 Pro with rustc 1.99 (Criterion medians; both sides write and read identical frames):

Verification

• Differential tests against zstd 1.5.7 (built from the vendored submodule by zstd-ffi). They cover every level on a standard corpus, thousands of random advanced-parameter configurations, streaming schedules with flushes and wrapped windows, LDM, short destinations, and a build mode that forces index overflow correction on both sides.
• Decoder tests: reference frames through every entry point, upstream's golden vectors, truncation and bit-flip corruption, limits.
• AFL++ targets in fuzz/afl. Encoder targets: differential, output capacity, sessions, lifecycle, parallel and parameter parsing. Decoder targets: raw input against the reference decoder, chunked sessions, and reference-written frames.
• 100% function coverage, checked with cargo llvm-cov.

See architecture/ for how the code is organized and verified.

Not supported

• Dictionaries
• Magicless frames
• targetCBlockSize
• External sequence producers
• Legacy (v0.x) frames
• Byte identity with zstd's multithreaded mode

License

MIT. The encoder is a port of zstd, used under its BSD license; see NOTICE.