Crate dataflow_nlp

Modules§

batching: Uilities for dealing with batches, such as shuffling and sorting batches
pipelines: Dataflow pipeline nodes
tokenization: All tokenization and untokenization
vocab: Vocab object and the functions to load different vocabularies