Struct DataLoader 

pub struct DataLoader<D> { /* private fields */ }

DataLoader that iterates a dataset with shuffling, batching, and prefetching.

Operates on CPU tensors only. Transfer the resulting batches to GPU with Tensor::to_device() in the training loop for efficient bulk transfer.

Lifecycle

let loader = DataLoader::new(dataset, batch_size, seed, &device);
for epoch in 0..num_epochs {
    for batch in loader.iter(epoch) {
        let batch = batch?;
        // batch.inputs: [batch_size, ...]
        // batch.targets: [batch_size, ...]
    }
}

Implementations

impl<D> DataLoader<D>

where D: Dataset<CpuRuntime>,

pub fn new(dataset: D, batch_size: usize, seed: u64, device: CpuDevice) -> Self

Create a new DataLoader.

Arguments

• dataset - The dataset to iterate
• batch_size - Number of samples per batch
• seed - Random seed for deterministic shuffling
• device - CPU device for tensor creation

pub fn with_prefetch(self, count: usize) -> Self

Set the number of batches to prefetch in the background thread.

Default is 2. Set to 0 to disable prefetching (synchronous iteration).

pub fn num_batches(&self) -> usize

Number of batches per epoch.

pub fn dataset(&self) -> &D

The underlying dataset.

pub fn iter(&self, epoch: u64) -> DataLoaderIter<'_, D>

Create a synchronous iterator for one epoch.

Indices are shuffled deterministically from seed + epoch. The last incomplete batch (if any) is dropped.

impl<D> DataLoader<D>

where D: Dataset<CpuRuntime> + Clone + 'static,

pub fn prefetch_iter(&self, epoch: u64) -> PrefetchIter

Create a prefetching iterator that loads batches in a background thread.

The background thread loads up to prefetch_count batches ahead. Requires D: Clone because the dataset is moved into the background thread.