Expand description
Zstandard encoder — frame compression, streaming, dictionary support.
Four entry points cover the common use cases:
compress— one-shot helper that builds a self-contained Zstandard frame from aReadsource to aWritesink. The input is consumed incrementally fromRead, so input buffering stays bounded; however, the compressed output is buffered in memory until the frame is complete so the Frame Content Size field can be filled in the header — peak memory isO(compressed_size)(worst-caseO(input_size)for incompressible payloads, plus a small frame overhead). The savings vscompress_to_veccome from not materialising the input alongside the output.compress_to_vec— same one-shot path ascompressbut the input is eagerly drained into an internalVecfirst (read_to_end) so the encoder can be handed a&[u8]and a precise source-size hint. Peak memory is therefore ≈input_size + output_size; prefercompressorStreamingEncoderwhen the input is large or unbounded.StreamingEncoder— implementscrate::io::Write, which re-exportsstd::io::Writeunder thestdfeature and falls back to ano_std-friendly trait otherwise. Accepts bytes incrementally and flushes compressed output as blocks fill. Requiresset_pledged_content_sizebefore the first write if the Frame Content Size field is to be populated.FrameCompressor— lower-level builder that owns the matcher and the per-frame configuration; the streaming and one-shot helpers are thin wrappers over it. Reach for it when you need to swap in a customMatcherimplementation or share the matcher across frames.
Compression intensity is selected via CompressionLevel, which
provides both named presets (Fastest, Default, Better, Best) and
numeric levels (from_level(n)) that mirror C zstd’s level numbering
(negative for ultra-fast, 0 = default, 1..=22 for the standard
range).
All produced frames are valid RFC 8878 Zstandard streams and decode
through both this crate’s crate::decoding module and upstream C zstd.
For memory budgeting, estimated_compression_workspace_bytes reports
the approximate steady-state heap footprint of a one-shot compression at
a given level (window + match-finder tables + block staging).
Re-exports§
pub use frame_emit_info::BlockType;lsmpub use frame_emit_info::FrameBlock;lsmpub use frame_emit_info::FrameEmitInfo;lsm
Modules§
- frame_
emit_ info lsm - Structural metadata describing the layout of an emitted zstd frame.
Structs§
- Bounds
- Inclusive
[lower_bound, upper_bound]range for aCParameter, the drop-in equivalent of C zstd’sZSTD_bounds. - Compression
Parameters - Fully-resolved fine-grained compression parameters. Build through
CompressionParameters::builder; pass toFrameCompressor::set_parametersorcompress_with_parameters. - Compression
Parameters Builder - Builder for
CompressionParameters. Each setter records one knob;Self::buildvalidates them againstCParameter::bounds. - Encoder
Dictionary - A dictionary prepared for the ENCODER side, analogous to zstd’s
CDict(vs the decoder’sDictionary/DDict). - Frame
Compressor - An interface for compressing arbitrary data with the ZStandard compression algorithm.
- Match
Generator Driver - This is the default implementation of the
Matchertrait. It allocates and reuses the buffers when possible. - Streaming
Encoder - Incremental frame encoder that implements
Write.
Enums§
- CParameter
- One tunable compression parameter — the analogue of a C zstd
ZSTD_cParameter. Used to query bounds viaCParameter::bounds. - Compression
Level - The compression mode used impacts the speed of compression, and resulting compression ratios. Faster compression will result in worse compression ratios, and vice versa.
- Parameter
Error - Error returned by
CompressionParametersBuilder::buildwhen a knob is set outside itsCParameter::bounds. - Sequence
- Sequences that a
Matchercan produce - Strategy
- Match-finder strategy — the drop-in equivalent of C zstd’s
ZSTD_strategyenum (ZSTD_fast…ZSTD_btultra2). The numeric ordinals match upstream (fast = 1…btultra2 = 9), soStrategy::ordinal/Strategy::from_ordinalround-trip with the CZSTD_c_strategyparameter value.
Traits§
- Matcher
- Trait used by the encoder that users can use to extend the matching facilities with their own algorithm making their own tradeoffs between runtime, memory usage and compression ratio
Functions§
- compress
- Convenience function to compress some source into a target without reusing any resources of the compressor
- compress_
bound - Worst-case compressed-frame size for an input of
src_sizebytes. - compress_
slice_ to_ vec - Compress a contiguous byte slice into a fresh
Vec<u8>without the input-buffering step thatcompress_to_vecperforms to adapt aReadsource. - compress_
to_ vec - Convenience function to compress some source into a Vec without reusing any resources of the compressor.
- compress_
with_ parameters - Compress a byte slice into a fresh
Vec<u8>using fine-grainedCompressionParameters(#27) instead of a bareCompressionLevel. - estimated_
bt_ strategy_ extra_ bytes - Extra steady-state workspace the binary-tree strategies (ordinals 6..=9, btlazy2..btultra2) retain beyond the hash/chain tables: the boxed matcher plus its scratch arenas, and the HC3 short-match side table for btultra/btultra2 (capped by the window log). 0 for non-BT ordinals.
- estimated_
compression_ workspace_ bytes - Estimated steady-state heap footprint of a one-shot compression context
at
level(window history + match-finder tables + block staging), in bytes. Computed from the same per-level tuning table the encoder resolves at frame start, so the estimate tracks the real allocations; it is an upper-bound style budget figure, not an exact accounting.