Expand description
Fast, pure-Rust zstd compression.
zrip implements zstd compression levels -7 through 4 (Fast and DFast strategies),
targeting high-speed compression for data transfers. It produces standard zstd
frames decompressible by any compliant decoder.
§Quick start
let data = b"hello world, hello zstd compression!";
// Compress at level 1 (fast)
let compressed = zrip::compress(data, 1).unwrap();
// Decompress
let decompressed = zrip::decompress(&compressed).unwrap();
assert_eq!(&decompressed, data);§Compression levels
| Level | Strategy | Notes |
|---|---|---|
| -7 | Fast | Fastest; raw literals and long matches only, for LZ4’s niche |
| -6..=-1 | Fast | Fast encode, lower ratio |
| 0 | Library default (currently level 1) | |
| 1..=2 | Fast | Good balance for network transfers |
| 3..=4 | DFast | Better ratio, still fast |
Level -8, a separate zrip level in earlier versions, runs as -7.
MIN_LEVEL, MAX_LEVEL and DEFAULT_LEVEL name the bounds and
the default.
§Streaming
FrameEncoder and FrameDecoder implement std::io::Write and
std::io::Read for streaming compression and decompression.
use std::io::{Write, Read};
let mut encoder = zrip::FrameEncoder::new(Vec::new(), 1).unwrap();
encoder.write_all(b"streaming data").unwrap();
let compressed = encoder.finish().unwrap();
let mut decoder = zrip::FrameDecoder::new(&compressed[..]);
let mut output = Vec::new();
decoder.read_to_end(&mut output).unwrap();
assert_eq!(&output, b"streaming data");push::PushEncoder and push::PushDecoder are the push-style
(sans-io) counterparts: feed input chunks, drain output into your own
buffers. They need only alloc.
§Long distance matching
For data with long-range repetitions, enable LDM via Options:
let opts = zrip::Options::default().window_log(24).ldm(true);
let compressed = zrip::compress_opts(b"data with long-range repeats", 1, &opts).unwrap();Without window_log, LDM uses a 128 MiB window. One-shot calls shrink it to
the input.
§Buffer reuse
For repeated compression/decompression, CompressContext and
DecompressContext reuse internal buffers across calls, reducing allocation
overhead in hot loops.
§Feature flags
std(default): enablesallocand standard library support.alloc:no_stdwith heap allocation (Vec, etc.).frame(default): frame header parsing/writing; impliesstd.ldm(default): long distance matching for large-window compression.dict_builder: FastCOVER dictionary training.simd(default): runtime SIMD dispatch (AVX2+BMI2, NEON, SIMD128).paranoid: keeps any futureunsafeout; no effect today.nightly:#[optimize]attributes for hot paths.
Modules§
- adaptive
- Compression level adaptation for a sender whose bottleneck moves.
- dict
- zstd dictionaries: parsing, and FastCOVER training with the
dict_builderfeature. - error
- Compression and decompression errors.
- frame
- Zstd frame format: constants, header parsing, and frame size scanning.
- push
- Push-style (sans-io) streaming: feed input chunks, drain output into
your own buffers. Needs only
alloc.
Structs§
- Compress
Context - Reusable compression context that amortizes hash table and buffer allocations.
- Decompress
Context - Reusable decompression context that amortizes buffer allocations.
- Dictionary
- A pre-trained zstd dictionary for improved compression of small data.
- Frame
Decoder - Streaming zstd decompressor implementing
Read. - Frame
Encoder - Streaming zstd compressor implementing
Write. - LdmParams
- Parameters for Long Distance Matching.
- Level
Params - Compression parameters for a specific level.
- Options
- Options for large-window and LDM compression, orthogonal to level.
Enums§
- Compress
Error - Error returned by compression functions.
- Decompress
Error - Error returned by decompression functions.
- Zstd
Error - Unified error type wrapping both compression and decompression errors.
Constants§
- DEFAULT_
DECOMPRESS_ LIMIT - Output limit of the calls that take none, such as
decompress: no limit. - DEFAULT_
LEVEL - Default compression level used when level 0 is requested.
- MAX_
LEVEL - Highest compression level.
- MIN_
LEVEL - Lowest compression level. Level -8, a separate level in earlier versions, is still accepted and runs as -7.
- SAFE_
DECOMPRESS_ LIMIT - A conservative output limit (128 MiB) suitable for decompressing untrusted input.
Functions§
- compress
- Compresses
inputinto one frame atlevel(seeMIN_LEVELandMAX_LEVEL; 0 meansDEFAULT_LEVEL). - compress_
bound - Worst-case frame size for
input_lenbytes of input, with or without a dictionary. - compress_
into compressintooutput. Returns the frame size. Fails withCompressError::OutputTooSmallif the frame does not fit;compress_boundis always enough.- compress_
opts compresswithOptions, such as a larger window and long distance matching.- compress_
with_ dict compresswithdict. Decoders need the same dictionary.- compress_
with_ params compresswith explicit match finder parameters instead of a level. Parameters larger than the input needs are reduced to fit it.- decompress
- Decompresses every frame in
input, skipping skippable frames. - decompress_
into decompress, appending tooutput. Returns the number of bytes appended. Reusingoutputsaves the allocation.- decompress_
with_ dict decompressfor frames compressed withdict.- decompress_
with_ dict_ and_ limit decompress_with_dictwith the output capped atmax_output_sizebytes. Larger output fails withDecompressError::OutputTooSmall.- decompress_
with_ limit decompresswith the output capped atmax_output_sizebytes. Larger output fails withDecompressError::OutputTooSmall.