Module data 

Re-exports

pub use dataset::Batch;

pub use dataset::Dataset;

pub use loader::DataLoader;

pub use mmap::MmapDataset;

pub use sharded::ShardedDataset;

Modules

collate

Batch collation and index shuffling helpers.

dataset

Dataset trait and batch type for training data pipelines.

loader

DataLoader with shuffling, batching, and background prefetching.

mmap

Memory-mapped dataset for large token files.

sharded

Sharded dataset wrapper for distributed training.