Struct SplitSampler 

pub struct SplitSampler { /* private fields */ }

Like RandomSampler, but an epoch is a slice of a data pass.

splits says how finely to cut one pass. The pass permutation is unchanged and still covers every sample exactly once; splitting only decides how much of it one epoch consumes, so splits epochs make one pass and no sample is seen twice along the way.

This is what makes single-pass training (the normal regime for LLM pretraining) workable: everything that keys off the epoch boundary — eval, checkpointing, reporting — gets a boundary to key off, where a naive one-epoch run has none until teardown.

use flodl::SplitSampler;

// One pass over 10k samples, delivered as 20 epochs of 500.
let sampler = SplitSampler::new(10_000, 42, 20);

At splits = 1 it behaves exactly like RandomSampler.

Implementations

impl SplitSampler

pub fn new(n: usize, seed: u64, splits: usize) -> Self

Create a split sampler for n samples with the given base seed.

splits is clamped to at least 1.

pub fn splits(&self) -> usize

Slices per data pass.

Trait Implementations

impl Sampler for SplitSampler

fn len(&self) -> usize

Total number of samples. Must match the dataset length.

fn epoch_len(&self) -> usize

How many indices one epoch visits.

fn indices(&mut self, epoch: usize) -> Vec<usize>

Generate the index ordering for a given epoch.

fn is_empty(&self) -> bool

Whether the sampler is empty.