Struct GroupedSampler 

pub struct GroupedSampler<F> { /* private fields */ }

Sampler that groups indices by a key function and samples groups together.

This sampler allows you to define custom grouping criteria and control how samples within and between groups are ordered. This is useful for scenarios where you want to process related samples together.

Use Cases

• Sequence Data: Group by sequence ID to process complete sequences
• Hierarchical Data: Group by category for structured processing
• Batch Constraints: Ensure certain samples appear in the same batch
• Memory Efficiency: Group similar samples for better cache locality

Implementations

impl<F> GroupedSampler<F>

where F: Fn(usize) -> usize + Send,

pub fn new<D>(dataset: &D, group_fn: F) -> Self

where D: Dataset,

Create a new grouped sampler.

Arguments

• dataset - Dataset to sample from
• group_fn - Function that maps sample index to group ID

Examples

use torsh_data::sampler::{Sampler, GroupedSampler};

let dataset = DummyDataset { len: 10 };

// Group by class (assuming 3 classes)
let group_by_class = |idx: usize| idx % 3;
let sampler = GroupedSampler::new(&dataset, group_by_class);

pub fn with_shuffle_groups(self, shuffle: bool) -> Self

Set whether to shuffle the order of groups.

Arguments

• shuffle - Whether to randomize group order

Examples

let dataset = DummyDataset { len: 10 };
let sampler = GroupedSampler::new(&dataset, |idx| idx % 3)
    .with_shuffle_groups(true);

pub fn with_shuffle_within_groups(self, shuffle: bool) -> Self

Set whether to shuffle within each group.

Arguments

• shuffle - Whether to randomize order within groups

Examples

let dataset = DummyDataset { len: 10 };
let sampler = GroupedSampler::new(&dataset, |idx| idx % 3)
    .with_shuffle_within_groups(true);

pub fn with_generator(self, seed: u64) -> Self

Set random generator seed.

Arguments

• seed - Seed for deterministic shuffling

pub fn num_groups(&self) -> usize

Get the number of groups.

pub fn group_sizes(&self) -> Vec<usize>

Get the sizes of all groups.

pub fn shuffles_groups(&self) -> bool

Check if groups will be shuffled.

pub fn shuffles_within_groups(&self) -> bool

Check if samples within groups will be shuffled.

Trait Implementations

impl<F: Debug> Debug for GroupedSampler<F>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<F: Send> Sampler for GroupedSampler<F>

type Iter = SamplerIterator

Iterator type returned by the sampler

fn iter(&self) -> Self::Iter

Create an iterator over indices

fn len(&self) -> usize

Total number of samples that will be yielded

fn is_empty(&self) -> bool

Check if sampler is empty

fn into_batch_sampler( self, batch_size: usize, drop_last: bool, ) -> BatchingSampler<Self>

where Self: Sized,

Convert this sampler into a batch sampler

fn into_distributed( self, num_replicas: usize, rank: usize, ) -> DistributedWrapper<Self>

where Self: Sized,

Create a distributed version of this sampler