Expand description
Mako is a high performance data handling library
Modules
Batching module contains several utilities for dealing with batches, such as shuffling and sorting batches
Dataloader module contains the main dataloader struct, as well as dataloader utilities
Pipeline module contains the dataflow pipeline struct, as well as all pipeline utilities
Tokenization module handles all tokenization and untokenization
Vocab module contains the mako vocab object and the functions to load different vocabularies