Function create_raw_dict_from_source 

pub fn create_raw_dict_from_source<R: Read, W: Write>(
    source: R,
    source_size: usize,
    output: &mut W,
    dict_size: usize,
)

Available on crate feature dict_builder only.

Read from source to create a “raw content” dictionary of dict_size. The completed dictionary is written to output.

• source will be used as training data for the entire dictionary.
• source_size influences how the data is divided and sampled and is measured in bytes. While this does not need to be exact, estimates should attempt to be larger than the actual collection size.
• output is where the completed dictionary will be written.
• dict_size determines how large the complete dictionary should be. The completed dictionary will be this size or smaller.

This function buffers the training source internally so sampling and epoch scoring can both inspect the same training data.