Crate dataflow_nlp

Source

Modulesยง

batching
Uilities for dealing with batches, such as shuffling and sorting batches
pipelines
Dataflow pipeline nodes
tokenization
All tokenization and untokenization
vocab
Vocab object and the functions to load different vocabularies