Skip to main content

Module encoding

Module encoding 

Source
Expand description

Zstandard encoder — frame compression, streaming, dictionary support.

Four entry points cover the common use cases:

  • compress — one-shot helper that builds a self-contained Zstandard frame from a Read source to a Write sink. The input is consumed incrementally from Read, so input buffering stays bounded; however, the compressed output is buffered in memory until the frame is complete so the Frame Content Size field can be filled in the header — peak memory is O(compressed_size) (worst-case O(input_size) for incompressible payloads, plus a small frame overhead). The savings vs compress_to_vec come from not materialising the input alongside the output.
  • compress_to_vec — same one-shot path as compress but the input is eagerly drained into an internal Vec first (read_to_end) so the encoder can be handed a &[u8] and a precise source-size hint. Peak memory is therefore ≈ input_size + output_size; prefer compress or StreamingEncoder when the input is large or unbounded.
  • StreamingEncoder — implements crate::io::Write, which re-exports std::io::Write under the std feature and falls back to a no_std-friendly trait otherwise. Accepts bytes incrementally and flushes compressed output as blocks fill. Requires set_pledged_content_size before the first write if the Frame Content Size field is to be populated.
  • FrameCompressor — lower-level builder that owns the matcher and the per-frame configuration; the streaming and one-shot helpers are thin wrappers over it. Reach for it when you need to swap in a custom Matcher implementation or share the matcher across frames.

Compression intensity is selected via CompressionLevel, which provides both named presets (Fastest, Default, Better, Best) and numeric levels (from_level(n)) that mirror C zstd’s level numbering (negative for ultra-fast, 0 = default, 1..=22 for the standard range).

All produced frames are valid RFC 8878 Zstandard streams and decode through both this crate’s crate::decoding module and upstream C zstd.

For memory budgeting, estimated_compression_workspace_bytes reports the approximate steady-state heap footprint of a one-shot compression at a given level (window + match-finder tables + block staging).

Re-exports§

pub use frame_emit_info::BlockType;lsm
pub use frame_emit_info::FrameBlock;lsm
pub use frame_emit_info::FrameEmitInfo;lsm

Modules§

frame_emit_infolsm
Structural metadata describing the layout of an emitted zstd frame.

Structs§

Bounds
Inclusive [lower_bound, upper_bound] range for a CParameter, the drop-in equivalent of C zstd’s ZSTD_bounds.
CompressionParameters
Fully-resolved fine-grained compression parameters. Build through CompressionParameters::builder; pass to FrameCompressor::set_parameters or compress_with_parameters.
CompressionParametersBuilder
Builder for CompressionParameters. Each setter records one knob; Self::build validates them against CParameter::bounds.
EncoderDictionary
A dictionary prepared for the ENCODER side, analogous to zstd’s CDict (vs the decoder’s Dictionary / DDict).
FrameCompressor
An interface for compressing arbitrary data with the ZStandard compression algorithm.
MatchGeneratorDriver
This is the default implementation of the Matcher trait. It allocates and reuses the buffers when possible.
StreamingEncoder
Incremental frame encoder that implements Write.

Enums§

CParameter
One tunable compression parameter — the analogue of a C zstd ZSTD_cParameter. Used to query bounds via CParameter::bounds.
CompressionLevel
The compression mode used impacts the speed of compression, and resulting compression ratios. Faster compression will result in worse compression ratios, and vice versa.
ParameterError
Error returned by CompressionParametersBuilder::build when a knob is set outside its CParameter::bounds.
Sequence
Sequences that a Matcher can produce
Strategy
Match-finder strategy — the drop-in equivalent of C zstd’s ZSTD_strategy enum (ZSTD_fastZSTD_btultra2). The numeric ordinals match upstream (fast = 1btultra2 = 9), so Strategy::ordinal / Strategy::from_ordinal round-trip with the C ZSTD_c_strategy parameter value.

Traits§

Matcher
Trait used by the encoder that users can use to extend the matching facilities with their own algorithm making their own tradeoffs between runtime, memory usage and compression ratio

Functions§

compress
Convenience function to compress some source into a target without reusing any resources of the compressor
compress_bound
Worst-case compressed-frame size for an input of src_size bytes.
compress_slice_to_vec
Compress a contiguous byte slice into a fresh Vec<u8> without the input-buffering step that compress_to_vec performs to adapt a Read source.
compress_to_vec
Convenience function to compress some source into a Vec without reusing any resources of the compressor.
compress_with_parameters
Compress a byte slice into a fresh Vec<u8> using fine-grained CompressionParameters (#27) instead of a bare CompressionLevel.
estimated_bt_strategy_extra_bytes
Extra steady-state workspace the binary-tree strategies (ordinals 6..=9, btlazy2..btultra2) retain beyond the hash/chain tables: the boxed matcher plus its scratch arenas, and the HC3 short-match side table for btultra/btultra2 (capped by the window log). 0 for non-BT ordinals.
estimated_compression_workspace_bytes
Estimated steady-state heap footprint of a one-shot compression context at level (window history + match-finder tables + block staging), in bytes. Computed from the same per-level tuning table the encoder resolves at frame start, so the estimate tracks the real allocations; it is an upper-bound style budget figure, not an exact accounting.