Struct DistributedSampler 

pub struct DistributedSampler<S> { /* private fields */ }

A source wrapper that shards samples across distributed ranks.

Only samples where index % world_size == rank are yielded. The source item’s position within the underlying iterator determines the index.

use tenshift_core::sample::Sample;
use tenshift_core::source::Source;
use tenshift_core::sources::{DistributedSampler, MemorySource};

let source = MemorySource::new("demo", vec![Sample::new(), Sample::new(), Sample::new()]);
let shard = DistributedSampler::new(source, 1, 2).unwrap();
assert_eq!(shard.len_hint(), Some(1));

Implementations

impl<S> DistributedSampler<S>

pub fn new(inner: S, rank: usize, world_size: usize) -> Result<Self>

where S: Source,

Create a distributed sampler over an existing source.

Errors

Returns Error::InvalidConfig when world_size == 0 or rank >= world_size.

Trait Implementations

impl<S> Source for DistributedSampler<S>

where S: Source,

fn open(&self) -> Result<Box<dyn SourceIterator>>

Create an iterator over samples from this source.

fn len_hint(&self) -> Option<u64>

Optional hint about the total number of samples.

fn name(&self) -> &str

Human-readable name for this source (for logging and error messages).