Struct ZstdDictionary 

pub struct ZstdDictionary { /* private fields */ }

Pre-trained zstd dictionary for improved compression of small blocks.

Zstd dictionaries significantly improve compression ratios for blocks in the 4–64 KiB range typical of LSM-trees, especially when data has recurring patterns (e.g., structured keys, repeated prefixes, JSON/MessagePack values).

The dictionary is identified by a 32-bit ID derived from its content (truncated xxh3 hash). This ID is stored alongside compressed blocks so readers can detect dictionary mismatches.

Example

use lsm_tree::ZstdDictionary;

let samples: &[u8] = &training_data;
let dict = ZstdDictionary::new(samples);

Implementations

impl ZstdDictionary

pub fn new(raw: &[u8]) -> Self

Creates a new dictionary handle from raw bytes.

raw may be either:

• A finalized zstd dictionary — bytes starting with the magic 37 A4 30 EC (as produced by zstd --train; accessible via ZstdDictionary::raw for persistence and interop). The backend parses it with the full entropy-table decoder.
• A raw content dictionary — arbitrary bytes used as LZ77 history (no magic header). Useful when the caller controls the training data and does not need the full entropy-table overhead.

Both forms are accepted by [CompressionProvider::compress_with_dict] and [CompressionProvider::decompress_with_dict].

The handle stores the full 64-bit xxh3 hash of raw internally. Self::id returns the lower 32 bits for external consumers (config validation, frame header); id64 (crate-internal) exposes the full fingerprint for use as a cache key.

pub fn id(&self) -> u32

Returns a 32-bit fingerprint derived from the dictionary content.

The fingerprint is the lower 32 bits of the xxh3-64 hash of the raw dictionary bytes. It is stable for a given byte sequence and is intended for config validation (matching a CompressionType::ZstdDict dict_id field against the supplied ZstdDictionary) and external interop.

The value may theoretically be 0 (probability ≈ 1/2³²). Backends that embed a dict ID in the zstd frame header (where id=0 is reserved) are responsible for clamping to at least 1 themselves. Config validation is unaffected: both sides derive the ID from the same bytes and therefore agree even in the zero case.

pub fn raw(&self) -> &[u8]

Returns the raw dictionary bytes.

Trait Implementations

impl Clone for ZstdDictionary

Available on zstd_any only.

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ZstdDictionary

Available on zstd_any only.

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for ZstdDictionary

Available on zstd_any only.

impl PartialEq for ZstdDictionary

Available on zstd_any only.

Two dictionaries are equal when their full 64-bit xxh3 fingerprints agree. Equality is defined by the 64-bit id field; hash collisions between dictionaries with different raw bytes are theoretically possible but extremely unlikely given the xxh3-64 collision probability.

fn eq(&self, other: &Self) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.