Function finalize_raw_dict 

pub fn finalize_raw_dict(
    raw_content: &[u8],
    samples: &[u8],
    sample_sizes: &[usize],
    dict_size: usize,
    options: FinalizeOptions,
) -> Result<Vec<u8>>

Available on crate feature dict-builder only.

Finalize raw dictionary content into a dictionary of at most dict_size bytes: magic, id, entropy tables, repeat offsets and the content (the reference’s ZDICT_finalizeDictionary).

The entropy tables are measured, not guessed: the first block of each sample is compressed with raw_content as a raw dictionary at FinalizeOptions::level, and the tables describe the literals and sequences those blocks produced. samples is every sample back to back and sample_sizes their lengths. Content that does not fit after the header is cut from the front: trainers place their best segments last.

Errors

InvalidInput when raw_content is empty, when dict_size is below MIN_TRAINED_DICT_SIZE or leaves less than eight bytes of content after the header, when the id is zero, or when sample_sizes does not add up to samples.len(); a size too small carries TrainingError::DictionaryTooSmall.

Examples

use structured_zstd::dictionary::{FinalizeOptions, finalize_raw_dict};

let mut samples = Vec::new();
let mut sizes = Vec::new();
for i in 0..50u32 {
    let line = format!("tenant=demo table=orders key={i} status=shipped\n");
    sizes.push(line.len());
    samples.extend_from_slice(line.as_bytes());
}
let content = b"tenant=demo table=orders key= status=shipped\n";
let dict = finalize_raw_dict(content, &samples, &sizes, 1024, FinalizeOptions::default())
    .unwrap();
assert!(dict.ends_with(content));