Crate mako

Mako is a high performance data handling library

Modules

batching

Batching module contains several utilities for dealing with batches, such as shuffling and sorting batches

dataloader

Dataloader module contains the main dataloader struct, as well as dataloader utilities

tokenization

Tokenization module handles all tokenization and untokenization

vocab

Vocab module contains the mako vocab object and the functions to load different vocabularies