pub struct EncoderDictionary { /* private fields */ }Expand description
A dictionary prepared for the ENCODER side, analogous to zstd’s CDict
(vs the decoder’s Dictionary / DDict).
It carries the entropy tables, content, and repeat-offset history the
compressor needs, but is a distinct type with no decode path: there is
no way to turn it into a DictionaryHandle
or feed it to a FrameDecoder. That keeps
the compress-only state (which may have been parsed without building the
decode lookup tables, see
set_dictionary_from_bytes)
from ever reaching the decode side — the encoder/decoder dictionary split
mirrors C zstd’s CDict / DDict.
Cloning one is a handle, not a copy: it is attached to a compressor by
value, so a dictionary serving many frames would otherwise have its parsed
tables and content duplicated for each of them — on exactly the path where
one dictionary is prepared once precisely to be used again and again.
The encoder entropy tables a dictionary seeds into each frame’s first block
(upstream zstd cdict->cBlockState) are built once, here, and shared by
every compressor the dictionary is attached to.
Implementations§
Source§impl EncoderDictionary
impl EncoderDictionary
Sourcepub fn from_dictionary(dictionary: Dictionary) -> Self
pub fn from_dictionary(dictionary: Dictionary) -> Self
Wrap an already-parsed Dictionary for
encoder use. A fully-decoded dictionary is valid here; only the encoder
entropy tables, content, and offset history are read. The CDict cParams
tier is keyed by the content length here; prefer Self::from_bytes
when the serialized blob is at hand — it keys the tier by the exact
serialized size as upstream ZSTD_createCDict does.
Sourcepub fn from_bytes(raw_dictionary: &[u8]) -> Result<Self, DictionaryDecodeError>
pub fn from_bytes(raw_dictionary: &[u8]) -> Result<Self, DictionaryDecodeError>
Parse a serialized dictionary blob for encoder use, skipping the decode
lookup-table build the encoder never reads (see
Dictionary::decode_dict_for_encoding). The encoder entropy tables — and
thus the emitted frame — are identical to a full parse.
Sourcepub fn from_serialized_or_raw_content(
raw_dictionary: &[u8],
) -> Result<Self, DictionaryDecodeError>
pub fn from_serialized_or_raw_content( raw_dictionary: &[u8], ) -> Result<Self, DictionaryDecodeError>
Load whichever kind of dictionary raw_dictionary holds, the way
zstd -D does: a serialized blob is parsed, anything else is taken as
raw content (see
Dictionary::from_serialized_or_raw_content).
Either way the blob’s own length is what the compression-parameter tier
is chosen by, which is why this exists rather than parsing and calling
Self::from_dictionary: that keys the tier on the content length, and
for a serialized dictionary the entropy tables in between can put the
two on opposite sides of a boundary.
Sourcepub fn heap_size(&self) -> usize
pub fn heap_size(&self) -> usize
Heap bytes the prepared dictionary holds: the shared allocation itself, the parsed dictionary’s content and tables, and the entropy tables it seeds. Clones share all of it, so this counts once however many compressors the dictionary is attached to.
§Examples
use structured_zstd::encoding::EncoderDictionary;
let dictionary =
EncoderDictionary::from_serialized_or_raw_content(b"raw content, no header").unwrap();
assert!(dictionary.heap_size() >= b"raw content, no header".len());Sourcepub fn exclusive_heap_size<'a, I>(handles: I) -> usize
pub fn exclusive_heap_size<'a, I>(handles: I) -> usize
Heap bytes that handles alone keep alive: each distinct dictionary
among them once, and only when no handle outside them shares it.
For an owner holding several handles, such as a context whose compressors each keep the dictionary they last ran with: what it reports should count one dictionary once, and none that someone else also holds.
§Examples
use structured_zstd::encoding::EncoderDictionary;
let dictionary = EncoderDictionary::from_serialized_or_raw_content(b"some shared history").unwrap();
let copy = dictionary.clone();
// Both handles are in the set: counted once.
assert_eq!(
EncoderDictionary::exclusive_heap_size([&dictionary, ©]),
dictionary.heap_size()
);
// `copy` is held elsewhere: nothing is this set's alone.
assert_eq!(EncoderDictionary::exclusive_heap_size([&dictionary]), 0);Sourcepub fn id(&self) -> u32
pub fn id(&self) -> u32
The dictionary id.
Zero is a raw-content dictionary, which has no header to carry an id.
Such a dictionary attaches like any other; what changes is the frame,
which omits the Dictionary_ID field rather than storing a zero, so a
decoder has to be handed the same bytes explicitly.
Trait Implementations§
Source§impl Clone for EncoderDictionary
impl Clone for EncoderDictionary
Source§fn clone(&self) -> EncoderDictionary
fn clone(&self) -> EncoderDictionary
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more