# mzstd
Zstandard (RFC 8878) compression and decompression in safe Rust.
[][crate]
[][api]
[][ci]
[](#license)
- **Byte-identical encoder.** For the same input and parameters,
single-threaded compression writes exactly the bytes of zstd 1.5.7:
one-shot calls match `ZSTD_compress2`, and sessions match
`ZSTD_compressStream2` driven with the same chunks, output capacities and
flushes. This holds for levels -131072 to 22, the explicit compression
parameters, long distance matching, checksums, content sizes, pledged sizes
and size hints.
- **Independent decoder** written from the format specification. It decodes
every frame zstd writes apart from dictionary frames, with bounded window,
input and output limits.
- **Parallel compression** with tasks you run on your own executor. Frames
are valid Zstandard frames and are identical for any task count, but they
are not byte-identical to zstd's multithreaded mode.
- **No `unsafe`, no dependencies on zstd.** It runs with `no_std` + `alloc`,
and there is a C ABI in `mzstd-ffi`.
```toml
[dependencies]
mzstd = "0.2"
```
## Usage
```rust
use mzstd::{Compressor, Decompressor, EncoderConfig, Level};
let mut encoder = Compressor::new(EncoderConfig::default().with_level(Level::try_from(19)?))?;
let compressed = encoder.compress(b"hello, hello, hello")?;
let restored = Decompressor::default().decompress(&compressed)?;
assert_eq!(restored, b"hello, hello, hello");
# Ok::<(), Box<dyn std::error::Error>>(())
```
Explicit parameters override the level just as `ZSTD_CCtx_setParameter` does:
```rust
use mzstd::{CompressionParams, Compressor, EncoderConfig, LdmParams, Level, LongDistanceMatching, Strategy};
let config = EncoderConfig::default()
.with_level(Level::try_from(9)?)
.with_checksum(true)
.with_params(CompressionParams::default().with_window_log(24).with_strategy(Strategy::Lazy2))
.with_long_distance_matching(LongDistanceMatching::Enabled(LdmParams::default()));
let mut encoder = Compressor::new(config)?;
# Ok::<(), Box<dyn std::error::Error>>(())
```
## Choose your API
Compression and decompression expose the same core I/O shapes. Pick the shape that
matches how your application already moves bytes. `ParallelCompressor` (in
`mzstd::compressor::parallel`) is a separate execution strategy for compression: it
splits one frame into independent segments and hands them to your executor as tasks,
either for a whole input at once or for input that arrives in chunks. It requires `std`.
| Return a new `Vec<u8>` | `Compressor::compress` | `Decompressor::decompress` |
| Append to an existing Vec | `compress_into` | `decompress_into` |
| Write into a caller-owned slice | `compress_to_slice`, `compress_to_uninit` | `decompress_to_slice` |
| Pull output through `std::io::Read` | `Compressor::reader` | `Decompressor::reader` |
| Push input through `std::io::Write` | `Compressor::writer` | `Decompressor::writer` |
| Drive input/output incrementally | `start` → `EncoderSession` | `start` → `DecoderSession` |
| Same, with the session owning the codec | `into_session` → `EncoderSessionOwned` | `into_session` → `DecoderSessionOwned` |
| Split one known input across your threads | `ParallelCompressor::prepare_slice` → `ScopedParallelBatch` | — |
| Stream chunked input across your threads | `ParallelCompressor::start` → `ParallelEncoderSession` + `ParallelTaskQueue` | — |
| Same, with the session owning the compressor | `ParallelCompressor::into_session` → `ParallelEncoderSessionOwned` | — |
| Push or pull through `std::io` across your threads | `ParallelCompressor::writer`, `ParallelCompressor::reader` | — |
## Features
| `compression` | yes | `mzstd::compressor` |
| `decompression` | yes | `mzstd::decompressor` |
| `std` | yes | `std::error::Error`, `mzstd::io`, parallel compression |
| `no_std` | no | alloc-only build; overrides `std` |
| `hotpath`, `hotpath-cpu`, `hotpath-alloc` | no | profiling anchors |
## Performance
### Our benchmarks
Compression writes byte-identical frames to zstd 1.5.7 and reaches about
1.000× its speed across the comparison corpus; decompression reaches 1.033×.
These medians include empty and tiny inputs. Compressible text remains slower.
<details>
<summary>Compression benchmark results and methodology</summary>
![Compression speed and output size relative to zstd 1.5.7 across levels -5 to 22][bench-chart]
**Recorded 2026-10-14 · Intel Core i7-13700KF · WSL2 · cold one-shot APIs.**
Each value is the median of per-dataset ratios across eight equally weighted datasets,
including empty and tiny inputs. A speed ratio above **1×** is faster than C; every
frame is byte-identical to zstd's, so the size ratio is **1×** at every level. The
median over all 192 dataset/level pairs is **1.000×**, with lower mean latency
in 96 pairs. The margin does not establish an overall lead. Across levels,
Alice text reaches a median **0.925×** and cyclic Alice at 1 MiB **0.934×**;
empty, tiny and repeated-byte inputs are faster than C.
The benchmark includes compressor construction, allocation, compression, and disposal.
Results apply to the [recorded revision and machine][bench-levels].
[Per-dataset results and methodology][benchmarks] · [Raw measurements][bench-csv] · [Reproduce the benchmarks][benchmarking]
</details>
<details>
<summary>Decompression benchmark results and methodology</summary>
![Decompression speed relative to zstd 1.5.7 by source level][decoder-bench-chart]
**Recorded 2026-10-13 · Intel Core i7-13700KF · WSL2 · cold one-shot APIs.**
zstd C and mzstd decode identical C-written frames at source levels **-5 to 22**.
All **384 cases** restore the original bytes. mzstd runs at **1.03–1.04×** C's speed at
every source level (median **1.033×** over all 192 pairs): much faster on empty and
tiny frames, faster on random and run-length data, and 0.81–0.96× on compressible
text. Construction, allocation, decode and disposal are timed; C reads the output
capacity from the frame header.
[Per-level and dataset results][decoder-benchmarks] · [Raw measurements][decoder-bench-csv] · [Methodology and limits][decoder-bench-report]
</details>
## Verification
- Differential tests against zstd 1.5.7 (built from the vendored submodule by
`zstd-ffi`). They cover every level on a standard corpus, thousands of
random advanced-parameter configurations, streaming schedules with flushes
and wrapped windows, LDM, short destinations, and a build mode that forces
index overflow correction on both sides.
- Decoder tests: reference frames through every entry point, upstream's
golden vectors, truncation and bit-flip corruption, limits.
- AFL++ targets in `fuzz/afl`. Encoder targets: differential, output
capacity, sessions, lifecycle, parallel and parameter parsing. Decoder
targets: raw input against the reference decoder, chunked sessions, and
reference-written frames.
- 100% function coverage, checked with `cargo llvm-cov`.
See [`architecture/`](architecture/README.md) for how the code is organized
and verified.
## Not supported
- Dictionaries
- Magicless frames
- `targetCBlockSize`
- External sequence producers
- Legacy (v0.x) frames
- Byte identity with zstd's multithreaded mode
## License
Licensed under either of
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or
<https://www.apache.org/licenses/LICENSE-2.0>)
- MIT license ([LICENSE-MIT](LICENSE-MIT) or
<https://opensource.org/licenses/MIT>)
at your option. The encoder is a port of zstd, used under its BSD license;
see [NOTICE](NOTICE).
Unless you explicitly state otherwise, any contribution intentionally
submitted for inclusion in this work by you, as defined in the Apache-2.0
license, shall be dual licensed as above, without any additional terms or
conditions.
[crate]: https://crates.io/crates/mzstd
[api]: https://docs.rs/mzstd
[ci]: https://github.com/M-Compressors/mzstd/actions/workflows/ci.yml
[benchmarks]: ./docs/benchmarks/README.md
[bench-levels]: ./docs/benchmarks/encoders/README.md
[bench-csv]: ./docs/benchmarks/encoder-comparison.csv
[bench-chart]: ./docs/benchmarks/encoders/charts/overview.svg
[benchmarking]: ./docs/benchmarking.md
[decoder-benchmarks]: ./docs/benchmarks/decoders/README.md
[decoder-bench-csv]: ./docs/benchmarks/decoder-comparison.csv
[decoder-bench-chart]: ./docs/benchmarks/decoders/charts/overview.svg
[decoder-bench-report]: ./docs/benchmarks/decoder-comparison.md