Skip to main content

create_raw_dict_from_dir

Function create_raw_dict_from_dir 

Source
pub fn create_raw_dict_from_dir<P: AsRef<Path>, W: Write>(
    path: P,
    output: &mut W,
    dict_size: usize,
) -> Result<(), Error>
Available on crate feature dict_builder only.
Expand description

Creates a “raw content” dictionary, training off of every file in this directory and all sub-directories.

The resulting dictionary will be approximately dict_size or less, and written to output.

§Errors

This function returns Ok(()) if the dictionary was created successfully, and an Err(io::Error) if an error was encountered reading the input directory or writing dictionary bytes to output.

§Examples

use std::fs::File;
// Create a roughly 1mb dictionary, training off of file in `sample_files`
let input_folder = "sample_files/";
let mut output = File::create("output.dict").unwrap();
structured_zstd::dictionary::create_raw_dict_from_dir(input_folder, &mut output, 1_000_000)
    .expect("dictionary training from sample_files should succeed");