Module dictionary 

Available on crate feature dict_builder only.

Code for creating a separate content dictionary.

Effective dictionaries are up to 1% the size of the complete training body, and are trained on many examples of the original data.

Implemented following the paper “Effective construction of Relative Lempel-Ziv Dictionaries”, by Kewen Liao, Matthias Petri, Alistair Moffat, and Anthony Wirth

Functions

create_raw_dict_from_dir

Creates a “raw content” dictionary, training off of every file in this directory and all sub-directories.

create_raw_dict_from_source

Read from source to create a “raw content” dictionary of dict_size. The completed dictionary is written to output.