Struct Dictionary 

pub struct Dictionary {
    pub id: u32,
    pub fse: Box<FSEScratch>,
    pub huf: HuffmanScratch,
    pub dict_content: Vec<u8>,
    pub offset_hist: [u32; 3],
}

Zstandard includes support for “raw content” dictionaries, that store bytes optionally used during sequence execution.

https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md#dictionary-format

Fields

id: u32

A 4 byte value used by decoders to check if they can use the correct dictionary.

Zero means unidentified: a raw-content dictionary has no header to carry an ID, and the frames built from it record none, so it can only be supplied explicitly and never resolved from a frame header. Registration by ID (FrameDecoder::add_dict) therefore still requires a non-zero one.

fse: Box<FSEScratch>

A dictionary can contain an entropy table, either FSE or Huffman. Boxed for the same reason as DecoderScratch::fse: the ~12 KiB of inline tables must not ride the stack through construction.

huf: HuffmanScratch

A dictionary can contain an entropy table, either FSE or Huffman.

dict_content: Vec<u8>

The content of a dictionary acts as a “past” in front of data to compress or decompress, so it can be referenced in sequence commands. As long as the amount of data decoded from this frame is less than or equal to Window_Size, sequence commands may specify offsets longer than the total length of decoded output so far to reference back to the dictionary, even parts of the dictionary with offsets larger than Window_Size. After the total output has surpassed Window_Size however, this is no longer allowed and the dictionary is no longer accessible

offset_hist: [u32; 3]

The 3 most recent offsets are stored so that they can be used during sequence execution, see https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md#repeat-offsets for more.

Implementations

impl Dictionary

pub fn heap_bytes(&self) -> usize

Heap bytes owned by this dictionary: the content plus the parsed entropy tables’ heap. fse is a Box, so the fixed-size FSE decode arrays are heap bytes rather than inline in size_of::<Dictionary>(); charge that size_of here so a caller budgeting against the reported figure sees them.

pub const fn max_heap_bytes(content_len: usize) -> usize

Most heap_bytes a dictionary holding content_len bytes of content can report, so a caller can budget for one before parsing it.

Examples

use structured_zstd::decoding::Dictionary;

let dict = Dictionary::from_raw_content(0, vec![7u8; 1024]).unwrap();
assert!(dict.heap_bytes() <= Dictionary::max_heap_bytes(1024));

pub fn from_raw_content( id: u32, dict_content: Vec<u8>, ) -> Result<Dictionary, DictionaryDecodeError>

Build a dictionary from raw content bytes (without entropy table sections).

This is primarily intended for dictionaries produced by the dict-builder module, which currently emits raw-content dictionaries.

An id of 0 means the dictionary is unidentified, which is what any plain file used as a dictionary is: frames built with it record no dictionary ID, so it can only ever be supplied explicitly, never resolved from a frame header. Registration by ID (FrameDecoder::add_dict) still requires a non-zero one, since the ID is the key it is stored under.

pub fn decode_dict(raw: &[u8]) -> Result<Dictionary, DictionaryDecodeError>

Parses the dictionary from raw, initializes its tables, and returns a fully constructed Dictionary whose id can be checked against the frame’s dict_id.

pub fn is_serialized(raw: &[u8]) -> bool

Whether raw is a serialized dictionary rather than raw content: it starts with MAGIC_NUM and has room for the ID after it. A shorter blob is raw content whatever it starts with, as upstream zstd reads it (zstd_ddict.c:97, zstd_compress.c:5206).

use structured_zstd::decoding::{Dictionary, DICTIONARY_MAGIC};
assert!(!Dictionary::is_serialized(&DICTIONARY_MAGIC));
assert!(Dictionary::is_serialized(&[0x37, 0xA4, 0x30, 0xEC, 7, 0, 0, 0]));

pub fn from_serialized_or_raw_content( raw: &[u8], ) -> Result<Dictionary, DictionaryDecodeError>

Loads whichever kind of dictionary raw holds, the way zstd -D does: a serialized dictionary with entropy tables and an ID, and anything else is taken as raw content, which is why any file can be handed to -D. A raw-content dictionary has no ID, so it must be supplied explicitly on both sides — see Self::from_raw_content.

pub fn into_handle(self) -> DictionaryHandle

Convert this parsed dictionary into a reusable shared handle.

Trait Implementations

impl AsRef<Dictionary> for DictionaryHandle

fn as_ref(&self) -> &Dictionary

Converts this type into a shared reference of the (usually inferred) input type.

impl Clone for Dictionary

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl From<Dictionary> for DictionaryHandle

fn from(dict: Dictionary) -> Self

Converts to this type from the input type.