Crate combiner

Structs

• Statistics
  Represents the statistics collected during the file combining process.

Functions

• combine_files
  Combines files from the specified directory into a single output file.
• get_bpe
  Returns the appropriate BPE based on the tokenizer name.
• print_statistics
  Prints the statistics of the file combining process in a formatted table.