Skip to main content

Crate zrip

Crate zrip 

Source
Expand description

Fast, pure-Rust zstd compression.

zrip implements zstd compression levels -7 through 4 (Fast and DFast strategies), targeting high-speed compression for data transfers. It produces standard zstd frames decompressible by any compliant decoder.

§Quick start

let data = b"hello world, hello zstd compression!";

// Compress at level 1 (fast)
let compressed = zrip::compress(data, 1).unwrap();

// Decompress
let decompressed = zrip::decompress(&compressed).unwrap();
assert_eq!(&decompressed, data);

§Compression levels

LevelStrategyNotes
-7FastFastest; raw literals and long matches only, for LZ4’s niche
-6..=-1FastFast encode, lower ratio
0Library default (currently level 1)
1..=2FastGood balance for network transfers
3..=4DFastBetter ratio, still fast

Level -8, a separate zrip level in earlier versions, runs as -7. MIN_LEVEL, MAX_LEVEL and DEFAULT_LEVEL name the bounds and the default.

§Streaming

FrameEncoder and FrameDecoder implement std::io::Write and std::io::Read for streaming compression and decompression.

use std::io::{Write, Read};

let mut encoder = zrip::FrameEncoder::new(Vec::new(), 1).unwrap();
encoder.write_all(b"streaming data").unwrap();
let compressed = encoder.finish().unwrap();

let mut decoder = zrip::FrameDecoder::new(&compressed[..]);
let mut output = Vec::new();
decoder.read_to_end(&mut output).unwrap();
assert_eq!(&output, b"streaming data");

push::PushEncoder and push::PushDecoder are the push-style (sans-io) counterparts: feed input chunks, drain output into your own buffers. They need only alloc.

§Long distance matching

For data with long-range repetitions, enable LDM via Options:

let opts = zrip::Options::default().window_log(24).ldm(true);
let compressed = zrip::compress_opts(b"data with long-range repeats", 1, &opts).unwrap();

Without window_log, LDM uses a 128 MiB window. One-shot calls shrink it to the input.

§Buffer reuse

For repeated compression/decompression, CompressContext and DecompressContext reuse internal buffers across calls, reducing allocation overhead in hot loops.

§Feature flags

  • std (default): enables alloc and standard library support.
  • alloc: no_std with heap allocation (Vec, etc.).
  • frame (default): frame header parsing/writing; implies std.
  • ldm (default): long distance matching for large-window compression.
  • dict_builder: FastCOVER dictionary training.
  • simd (default): runtime SIMD dispatch (AVX2+BMI2, NEON, SIMD128).
  • paranoid: keeps any future unsafe out; no effect today.
  • nightly: #[optimize] attributes for hot paths.

Modules§

adaptive
Compression level adaptation for a sender whose bottleneck moves.
dict
zstd dictionaries: parsing, and FastCOVER training with the dict_builder feature.
error
Compression and decompression errors.
frame
Zstd frame format: constants, header parsing, and frame size scanning.
push
Push-style (sans-io) streaming: feed input chunks, drain output into your own buffers. Needs only alloc.

Structs§

CompressContext
Reusable compression context that amortizes hash table and buffer allocations.
DecompressContext
Reusable decompression context that amortizes buffer allocations.
Dictionary
A pre-trained zstd dictionary for improved compression of small data.
FrameDecoder
Streaming zstd decompressor implementing Read.
FrameEncoder
Streaming zstd compressor implementing Write.
LdmParams
Parameters for Long Distance Matching.
LevelParams
Compression parameters for a specific level.
Options
Options for large-window and LDM compression, orthogonal to level.

Enums§

CompressError
Error returned by compression functions.
DecompressError
Error returned by decompression functions.
ZstdError
Unified error type wrapping both compression and decompression errors.

Constants§

DEFAULT_DECOMPRESS_LIMIT
Output limit of the calls that take none, such as decompress: no limit.
DEFAULT_LEVEL
Default compression level used when level 0 is requested.
MAX_LEVEL
Highest compression level.
MIN_LEVEL
Lowest compression level. Level -8, a separate level in earlier versions, is still accepted and runs as -7.
SAFE_DECOMPRESS_LIMIT
A conservative output limit (128 MiB) suitable for decompressing untrusted input.

Functions§

compress
Compresses input into one frame at level (see MIN_LEVEL and MAX_LEVEL; 0 means DEFAULT_LEVEL).
compress_bound
Worst-case frame size for input_len bytes of input, with or without a dictionary.
compress_into
compress into output. Returns the frame size. Fails with CompressError::OutputTooSmall if the frame does not fit; compress_bound is always enough.
compress_opts
compress with Options, such as a larger window and long distance matching.
compress_with_dict
compress with dict. Decoders need the same dictionary.
compress_with_params
compress with explicit match finder parameters instead of a level. Parameters larger than the input needs are reduced to fit it.
decompress
Decompresses every frame in input, skipping skippable frames.
decompress_into
decompress, appending to output. Returns the number of bytes appended. Reusing output saves the allocation.
decompress_with_dict
decompress for frames compressed with dict.
decompress_with_dict_and_limit
decompress_with_dict with the output capped at max_output_size bytes. Larger output fails with DecompressError::OutputTooSmall.
decompress_with_limit
decompress with the output capped at max_output_size bytes. Larger output fails with DecompressError::OutputTooSmall.