Docs.rs
  • dataflow-0.2.1
    • dataflow 0.2.1
    • Docs.rs crate page
    • MIT OR Apache-2.0
    • Links
    • crates.io
    • Source
    • Owners
    • jafioti
    • Dependencies
      • indicatif ^0.15 normal
      • itertools ^0.9 normal
      • lentrait ^0.2.3 normal
      • rand ^0.8.3 normal
      • regex ^1.5.5 normal
      • serde ^1.0 normal
      • serde_json ^1.0.59 normal
      • thread-control ^0.1.2 normal
      • tokenizers ^0.11.0 normal
      • tempfile ^3.1 dev
    • Versions
    • 73.08% of the crate is documented
  • Go to latest version
  • Platform
    • x86_64-unknown-linux-gnu
  • Feature flags
  • docs.rs
    • About docs.rs
    • Badges
    • Builds
    • Metadata
    • Shorthand URLs
    • Download
    • Rustdoc JSON
    • Build queue
    • Privacy policy
  • Rust
    • Rust website
    • The Book
    • Standard Library API Reference
    • Rust by Example
    • The Cargo Guide
    • Clippy Documentation
logo

logo

Crate dataflow

  • Structs
  • Traits
  • Functions
?
Change settings

List of all items

Structs

  • dataloader::Dataloader
  • dataloader::LenIterDataloader
  • pipeline::ArrayBatch
  • pipeline::Batch
  • pipeline::BatchStateless
  • pipeline::Connector
  • pipeline::Duplicator
  • pipeline::FileLoader
  • pipeline::KeyedLoader
  • pipeline::MapReduce
  • pipeline::Pair
  • pipeline::RandomLoader
  • pipeline::SingleStateless
  • pipeline::Sort
  • pipeline::Stateful
  • tokenization::AlphabetTokenizer
  • tokenization::BPETokenizer
  • tokenization::SentenceTokenizer
  • tokenization::WhitespaceTokenizer
  • tokenization::WordpieceTokenizer
  • vocab::BPEVocab
  • vocab::TokenNotFoundError
  • vocab::WordPieceVocab

Traits

  • pipeline::Node
  • tokenization::Tokenizer
  • vocab::Vocab

Functions

  • batching::filter_by_length
  • batching::pad_batch
  • batching::pad_mask
  • batching::shuffle_lists
  • batching::sort_lists_by_length