zrip

Pure Rust zstd codec. Levels -7 through 4 (Fast and DFast strategies). Optimized for encode throughput in transfer pipelines that need standard zstd frames at high speed.

zrip = "0.2"

Why zrip

Fastest pure-Rust zstd encoder available. 72% faster encode than structured-zstd 0.0.40 at L3, 33% faster at L1. Faster decode than structured-zstd at L3 (40%) and L-1 (21%). 2.9x faster decode than ruzstd 0.8 at L1.

Negative levels (-7..-1) for high-throughput pipelines. Most zstd libraries only expose levels 1+.

no_std + alloc. Works in embedded and kernel contexts with the alloc feature; frame requires std.

Dictionary compression. COVER and FastCOVER training built in for small-message workloads (log lines, JSON records, RPC payloads).

Performance

Geomean across a 15-file Silesia + misc corpus on Intel i7-8700B (x86_64, SSE2/AVX2), performance governor, turbo off. Ratio is original / compressed; higher is better.

zrip vs C zstd 1.5.7

Compressible (12-file geomean: Silesia text, XML, JSON, PDF, binaries)

Encode is 72-82% of C zstd, decode 52-75%. Ratio trails C zstd by ~10% at L1-L4. The gap is pure Rust vs hand-tuned C with SIMD assembly.

Incompressible (3-file geomean: SAO star catalog, X-ray, MRI)

Encode is 79% of C zstd at negative levels, closing to parity at L3-L4. Decode is 74-96% of C zstd. Both codecs produce near-1.0x ratios, so throughput is the only differentiator here.

API

// One-shot (allocating)
let compressed = zrip::compress(input, 1)?;
let original   = zrip::decompress(&compressed)?;

// One-shot into caller buffer
let n = zrip::compress_into(input, &mut output_buf, 1)?;
zrip::decompress_into(&compressed, &mut output_vec)?;

// Reusable context (amortizes table allocation across calls)
let mut ctx = zrip::CompressContext::new(1)?;
let compressed = ctx.compress(input)?;

let mut dec = zrip::DecompressContext::new();
let original = dec.decompress(&compressed)?;

Streaming

use std::io::Write;

let mut enc = zrip::FrameEncoder::new(Vec::new(), 1)?;
enc.write_all(b"hello")?;
enc.write_all(b" world")?;
let compressed = enc.finish()?;

use std::io::Read;
let mut dec = zrip::FrameDecoder::new(&compressed[..]);
let mut out = String::new();
dec.read_to_string(&mut out)?;

FrameEncoder and FrameDecoder own persistent hash tables and workspace buffers. Call reset() to start a new frame while reusing all allocations:

let mut enc = zrip::FrameEncoder::new(Vec::new(), 1)?;
enc.write_all(b"first frame")?;
let first  = enc.reset(Vec::new())?;  // finishes frame, keeps buffers
enc.write_all(b"second frame")?;
let second = enc.finish()?;

Dictionary compression

let dict = zrip::Dictionary::from_bytes(&dict_bytes)?;
let compressed = zrip::compress_with_dict(input, 1, &dict)?;
let original   = zrip::decompress_with_dict(&compressed, &dict)?;

Streaming with a dictionary:

let mut enc = zrip::FrameEncoder::with_dict(Vec::new(), 1, dict.clone())?;
enc.write_all(input)?;
let compressed = enc.finish()?;

let mut dec = zrip::FrameDecoder::with_dict(&compressed[..], dict);
let mut output = Vec::new();
dec.read_to_end(&mut output)?;

CompressContext::with_dict() and DecompressContext::with_dict() provide the same reuse for one-shot compression.

Features

Safety

zrip uses unsafe for performance, not a zero-unsafe codebase. All algorithm and control-flow code is #![forbid(unsafe_code)]. Unsafe is confined to small, auditable leaf modules:

• primitives.rs modules in bitstream/, huffman/, encode/, decode/: #[inline(always)] wrappers around get_unchecked, read_unaligned, set_len, and copy_nonoverlapping with debug_assert! guards.
• simd/ and simd_decode/: intrinsics and raw pointer arithmetic for wildcopy, copy-match, and the fused SIMD sequence decoder.
• huffman/decode_4stream.rs: pointer-based interleaved 4-stream Huffman decoder.

Levels

Level 0 maps to the library default (currently level 1).

L-7 skips Huffman table construction and always emits raw literal blocks with predefined FSE tables. This eliminates the most expensive part of the encode pipeline (Huffman tree build, stream encoding, custom FSE table estimation) at the cost of compression ratio. The result is a valid zstd frame that any decoder handles, but with LZ4-class encode throughput.

L-6 through L2 use the full encode pipeline: Huffman-compressed literals (with treeless reuse across blocks) and predefined or custom FSE tables for sequences, whichever produces smaller output.

L3 and L4 use the DFast strategy with two hash tables (short 4-byte and long 8-byte matches) for better match quality at lower throughput.