Struct DataLoader 

pub struct DataLoader<D, S, C> { /* private fields */ }

DataLoader for batching and iterating over datasets

The DataLoader provides an efficient way to iterate over datasets in batches, with support for parallel loading, shuffling, and various optimization strategies.

Type Parameters

• D: Dataset type implementing the Dataset trait
• S: Sampler type implementing the BatchSampler trait
• C: Collate function type implementing the Collate trait

Examples

use torsh_data::dataloader::core::DataLoader;
use torsh_data::dataset::TensorDataset;

let dataset = TensorDataset::new(vec![1, 2, 3, 4, 5]);
let dataloader = DataLoader::builder(dataset)
    .batch_size(2)
    .num_workers(4)
    .build()?;

for batch in dataloader.iter() {
    // Process batch
}

Implementations

impl<D: Dataset> DataLoader<D, (), ()>

pub fn builder(dataset: D) -> DataLoaderBuilder<D>

Create a new DataLoader builder

This provides a fluent API for configuring DataLoader options.

Arguments

• dataset - The dataset to iterate over

Returns

A DataLoaderBuilder for configuring the DataLoader

impl<D, S, C> DataLoader<D, S, C>

where D: Dataset, S: BatchSampler, C: Collate<D::Item>,

pub fn iter(&self) -> DataLoaderIterator<'_, D, S, C>

Create an iterator over the dataset

Returns a DataLoaderIterator that will yield batches according to the configured sampler and collation function.

pub fn len(&self) -> usize

Get the number of batches

Returns the total number of batches that will be produced by this DataLoader, based on the underlying sampler’s batch count.

pub fn is_empty(&self) -> bool

Check if the dataloader is empty

Returns true if the DataLoader will produce zero batches.

pub fn dataset(&self) -> &D

Get the dataset

pub fn sampler(&self) -> &S

Get the sampler

pub fn collate_fn(&self) -> &C

Get the collate function

pub fn num_workers(&self) -> usize

Get the number of workers

Trait Implementations

impl<D, S, C> DataLoaderTrait<D, C> for DataLoader<D, S, C>

where D: Dataset + Sync, S: BatchSampler + Sync, C: Collate<D::Item> + Sync, D::Item: Send, C::Output: Send, S::Iter: Iterator<Item = Vec<usize>>,

fn len(&self) -> usize

Get the number of batches

fn is_empty(&self) -> bool

Check if the dataloader is empty