Skip to main content

EncoderDictionary

Struct EncoderDictionary 

Source
pub struct EncoderDictionary { /* private fields */ }
Expand description

A dictionary prepared for the ENCODER side, analogous to zstd’s CDict (vs the decoder’s Dictionary / DDict).

It carries the entropy tables, content, and repeat-offset history the compressor needs, but is a distinct type with no decode path: there is no way to turn it into a DictionaryHandle or feed it to a FrameDecoder. That keeps the compress-only state (which may have been parsed without building the decode lookup tables, see set_dictionary_from_bytes) from ever reaching the decode side — the encoder/decoder dictionary split mirrors C zstd’s CDict / DDict. Cloning one is a handle, not a copy: it is attached to a compressor by value, so a dictionary serving many frames would otherwise have its parsed tables and content duplicated for each of them — on exactly the path where one dictionary is prepared once precisely to be used again and again.

The encoder entropy tables a dictionary seeds into each frame’s first block (upstream zstd cdict->cBlockState) are built once, here, and shared by every compressor the dictionary is attached to.

Implementations§

Source§

impl EncoderDictionary

Source

pub fn from_dictionary(dictionary: Dictionary) -> Self

Wrap an already-parsed Dictionary for encoder use. A fully-decoded dictionary is valid here; only the encoder entropy tables, content, and offset history are read. The CDict cParams tier is keyed by the content length here; prefer Self::from_bytes when the serialized blob is at hand — it keys the tier by the exact serialized size as upstream ZSTD_createCDict does.

Source

pub fn from_bytes(raw_dictionary: &[u8]) -> Result<Self, DictionaryDecodeError>

Parse a serialized dictionary blob for encoder use, skipping the decode lookup-table build the encoder never reads (see Dictionary::decode_dict_for_encoding). The encoder entropy tables — and thus the emitted frame — are identical to a full parse.

Source

pub fn from_serialized_or_raw_content( raw_dictionary: &[u8], ) -> Result<Self, DictionaryDecodeError>

Load whichever kind of dictionary raw_dictionary holds, the way zstd -D does: a serialized blob is parsed, anything else is taken as raw content (see Dictionary::from_serialized_or_raw_content).

Either way the blob’s own length is what the compression-parameter tier is chosen by, which is why this exists rather than parsing and calling Self::from_dictionary: that keys the tier on the content length, and for a serialized dictionary the entropy tables in between can put the two on opposite sides of a boundary.

Source

pub fn heap_size(&self) -> usize

Heap bytes the prepared dictionary holds: the shared allocation itself, the parsed dictionary’s content and tables, and the entropy tables it seeds. Clones share all of it, so this counts once however many compressors the dictionary is attached to.

§Examples
use structured_zstd::encoding::EncoderDictionary;

let dictionary =
    EncoderDictionary::from_serialized_or_raw_content(b"raw content, no header").unwrap();
assert!(dictionary.heap_size() >= b"raw content, no header".len());
Source

pub fn exclusive_heap_size<'a, I>(handles: I) -> usize
where I: IntoIterator<Item = &'a EncoderDictionary>, I::IntoIter: Clone,

Heap bytes that handles alone keep alive: each distinct dictionary among them once, and only when no handle outside them shares it.

For an owner holding several handles, such as a context whose compressors each keep the dictionary they last ran with: what it reports should count one dictionary once, and none that someone else also holds.

§Examples
use structured_zstd::encoding::EncoderDictionary;

let dictionary = EncoderDictionary::from_serialized_or_raw_content(b"some shared history").unwrap();
let copy = dictionary.clone();
// Both handles are in the set: counted once.
assert_eq!(
    EncoderDictionary::exclusive_heap_size([&dictionary, &copy]),
    dictionary.heap_size()
);
// `copy` is held elsewhere: nothing is this set's alone.
assert_eq!(EncoderDictionary::exclusive_heap_size([&dictionary]), 0);
Source

pub fn id(&self) -> u32

The dictionary id.

Zero is a raw-content dictionary, which has no header to carry an id. Such a dictionary attaches like any other; what changes is the frame, which omits the Dictionary_ID field rather than storing a zero, so a decoder has to be handed the same bytes explicitly.

Trait Implementations§

Source§

impl Clone for EncoderDictionary

Source§

fn clone(&self) -> EncoderDictionary

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.