Skip to main content

Crate libzstd_bitexact_rs

Crate libzstd_bitexact_rs 

Source
Expand description

A pure-Rust reimplementation of Zstandard, built to be bit-exact with the reference C implementation.

Decompression is complete: dictionaries (raw-content and trained/ZDICT), a configurable windowLogMax, and a Read-based StreamDecoder with a bounded sliding window. compress produces frames byte-identical to C libzstd 1.5.7 at every level (1–22 and the negative levels), and StreamEncoder mirrors ZSTD_compressStream2 flush/end behavior (see its docs for the current streaming length scope). Every table and loop is a faithful port of its counterpart in the C sources, and the crate is continuously differential-tested against the real libzstd — see ROADMAP.md for what remains.

decompress and decompress_with_limit cover the common one-shot cases; DecodeOptions composes an output limit, a maximum window log, and a Dictionary; StreamDecoder decodes incrementally from any reader.

// A tiny handcrafted frame: single-segment, content size 5, one RLE
// block repeating `a` five times.
let frame = [0x28, 0xB5, 0x2F, 0xFD, 0x20, 0x05, 0x2B, 0x00, 0x00, b'a'];
assert_eq!(libzstd_bitexact_rs::decompress(&frame).unwrap(), b"aaaaa");

Structs§

DecodeOptions
Configurable decompression: an output-size limit, a maximum accepted window log, and an optional dictionary.
Dictionary
A parsed Zstandard dictionary, ready to seed decompression.
StreamDecoder
A streaming Zstandard decompressor over a Read source.
StreamEncoder
Streaming Zstandard encoder (ZSTD_compressStream2 semantics).

Enums§

Error
Errors produced while decoding a Zstandard stream.

Constants§

WINDOW_LOG_MAX
ZSTD_WINDOWLOG_MAX on 64-bit targets: the largest window log the format permits, and the default ceiling DecodeOptions enforces. It is also the hard cap — raising DecodeOptions::window_log_max above it has no effect, since larger windows are invalid regardless.

Functions§

compress
ZSTD_compress: one-shot frame compression with the simple-API defaults (contentSize known and flagged, no checksum, no dictionary).
compress_mt
ZSTD_compress2 with nbWorkers >= 1 (multithreaded / job-splitting mode). Bit-exact with C libzstd 1.5.7’s MT output, reproduced single-threaded: the input is split into jobSize-byte jobs, each compressed with the previous job’s overlap tail as a raw-content prefix and reset repcodes, then the per-job block streams are concatenated into one frame.
compress_mt_with_dict
ZSTD_compress2 with nbWorkers >= 1 and a loaded dictionary — the one-shot multithreaded dict frame (ZSTD_compress2 after ZSTD_CCtx_loadDictionary with NbWorkers set). Like compress_mt but the dictionary is applied to job 0 only (zstdmt jobs[0].cdict); later jobs are plain overlap jobs. A one-shot frame pledges the known srcSize, which — once MT engages (> 512 KiB) — always exceeds every attach cutoff, so job 0 takes the CDict copy path (ZSTD_resetCCtx_byCopyingCDict), byte-identical to the streaming copy path.
compress_with_cdict
ZSTD_compress_usingCDict (what zstd::bulk::Compressor::with_dictionary uses): one-shot compression with the dictionary loaded as a CDict (Path B). Produces different bytes than compress_with_dict (Path A): the CDict tables are filled dtlm_full (tagged short cache), and the working context either attaches the CDict (small inputs ≤ the strategy cutoff) or copies its de-tagged tables (larger inputs).
compress_with_dict
ZSTD_compress_usingDict: one-shot frame compression primed with a dictionary, so the start of src can reference dict. Bit-exact with C libzstd 1.5.7 for the supported scope; unsupported configurations return Error::Encode rather than diverging.
decompress
Decompress a sequence of Zstandard frames (and/or skippable frames).
decompress_with_limit
Like decompress, but fails with Error::OutputTooLarge once the output would exceed limit bytes. Use this on untrusted input to defuse decompression bombs.