zrip

Pure Rust zstd codec. Levels -7 through 4 (Fast and DFast strategies). Optimized for encode throughput in transfer pipelines that need standard zstd frames at high speed.

Why zrip

Fastest pure-Rust zstd encoder available. 72% faster encode than structured-zstd 0.0.40 at L3, 33% faster at L1. Faster decode than structured-zstd at L3 (40%) and L-1 (21%). 2.9x faster decode than ruzstd 0.8 at L1.

Negative levels (-7..-1) for high-throughput pipelines. Most zstd libraries only expose levels 1+.

no_std + alloc. Works in embedded and kernel contexts with the alloc feature; frame requires std.

Dictionary compression. COVER and FastCOVER training built in for small-message workloads (log lines, JSON records, RPC payloads).

API

// One-shot (allocating)
let compressed = zrip::compress(input, 1)?;
let original   = zrip::decompress(&compressed)?;

// One-shot into caller buffer
let n = zrip::compress_into(input, &mut output_buf, 1)?;
zrip::decompress_into(&compressed, &mut output_vec)?;

// Reusable context (amortizes table allocation across calls)
let mut ctx = zrip::CompressContext::new(1)?;
let compressed = ctx.compress(input)?;

let mut dec = zrip::DecompressContext::new();
let original = dec.decompress(&compressed)?;

Streaming

use std::io::Write;

let mut enc = zrip::FrameEncoder::new(Vec::new(), 1)?;
enc.write_all(b"hello")?;
enc.write_all(b" world")?;
let compressed = enc.finish()?;

use std::io::Read;
let mut dec = zrip::FrameDecoder::new(&compressed[..]);
let mut out = String::new();
dec.read_to_string(&mut out)?;

FrameEncoder and FrameDecoder own persistent hash tables and workspace buffers. Call reset() to start a new frame while reusing all allocations:

let mut enc = zrip::FrameEncoder::new(Vec::new(), 1)?;
enc.write_all(b"first frame")?;
let first  = enc.reset(Vec::new())?;  // finishes frame, keeps buffers
enc.write_all(b"second frame")?;
let second = enc.finish()?;

Dictionary compression

let dict = zrip::Dictionary::from_bytes(&dict_bytes)?;
let compressed = zrip::compress_with_dict(input, 1, &dict)?;
let original   = zrip::decompress_with_dict(&compressed, &dict)?;

Streaming with a dictionary:

let mut enc = zrip::FrameEncoder::with_dict(Vec::new(), 1, dict.clone())?;
enc.write_all(input)?;
let compressed = enc.finish()?;

let mut dec = zrip::FrameDecoder::with_dict(&compressed[..], dict);
let mut output = Vec::new();
dec.read_to_end(&mut output)?;

CompressContext::with_dict() and DecompressContext::with_dict() provide the same reuse for one-shot compression.

Dictionary training

Build a dictionary from sample data using the built-in FastCOVER trainer. Requires the dict_builder feature.

use zrip::dict::{train_dict_fastcover, fastcover::FastCoverParams};

let samples: Vec<&[u8]> = messages.iter().map(|m| m.as_bytes()).collect();
let dict = train_dict_fastcover(&samples, 16384, FastCoverParams::default());
// Use with compress_with_dict() / decompress_with_dict()

Features

Safety

SAFETY.md documents the unsafe boundary and catalogs C zstd memory safety bugs that Rust prevents by construction.

All codec paths are fuzz-tested (16 targets, ~10.7M executions) and verified under Miri on both x86_64 and aarch64. Fuzz targets cover round-trip correctness, cross-validation against C zstd, streaming, dictionary modes, and corruption resistance (bitflip, splice, truncate, overwrite).

Design

DESIGN.md covers the encode/decode pipeline, SIMD dispatch, compile-time specialization, and divergences from C zstd.

Levels

Level 0 maps to the library default (currently level 1). See DESIGN.md for parameter details and pipeline behavior per level.