Struct ConIterOfIter

pub struct ConIterOfIter<I>
where
    I: Iterator,
    I::Item: Send + Sync,
{ /* private fields */ }

Concurrent iterator of a any generic type implementing a regular Iterator.

It can be created by calling iter_into_con_iter on any iterator.

This iterator has a fundamental difference from all other concurrent iterators in the following:

• Concurrent iterators in general allow for concurrent access to different elements of the source code without blocking each other;
• however, concurrent iterator of a generic iterator requires to serialize generation of elements which might lead pulling threads to wait each other.

This has the following implications:

• Whenever possible, it is better to create the concurrent iterator on the concrete type rather than the generic iterator.
• Still, the transformed concurrent iterator allows for a very convenient way to safely share the iterator among multiple threads, simply by a shared reference.
• Furthermore, for programs where the task performed on each element of the iterator is large enough, the overhead might be considered tolerable.

Examples

use orx_concurrent_iter::*;

let num_threads = 4;

let data: Vec<_> = (0..100).map(|x| x.to_string()).collect();

// an arbitrary iterator
let iter = data
    .into_iter()
    .filter(|x| !x.starts_with('3'))
    .map(|x| format!("{x}!"));

// converted into a concurrent iterator and shared with multiple threads
let con_iter = iter.iter_into_con_iter();

let process = |_x: String| { /* assume actual work */ };

std::thread::scope(|s| {
    for _ in 0..num_threads {
        s.spawn(|| {
            while let Some(value) = con_iter.next() {
                assert!(!value.starts_with('3') && value.ends_with('!'));
                process(value);
            }
        });
    }
});

Trait Implementations

impl<I> ConcurrentIter for ConIterOfIter<I>

where I: Iterator, I::Item: Send + Sync,

type Item = <I as Iterator>::Item

Type of the element that the concurrent iterator yields.

type SequentialIter = I

Type of the sequential iterator that the concurrent iterator can be converted into using the into_seq_iter method.

type ChunkPuller<'i> = ChunkPullerOfIter<'i, I> where Self: 'i

Type of the chunk puller that can be created using the chunk_puller method.

fn into_seq_iter(self) -> Self::SequentialIter

Converts the concurrent iterator into its sequential regular counterpart. Note that the sequential iterator is a regular Iterator, and hence, does not have any overhead related with atomic states. Therefore, it is useful where the program decides to iterate over a single thread rather than concurrently by multiple threads.

fn skip_to_end(&self)

Immediately jumps to the end of the iterator, skipping the remaining elements.

fn next(&self) -> Option<Self::Item>

Returns the next element of the iterator. It returns None if there are no more elements left.

fn next_with_idx(&self) -> Option<(usize, Self::Item)>

Returns the next element of the iterator together its index. It returns None if there are no more elements left.

fn size_hint(&self) -> (usize, Option<usize>)

Returns the bounds on the remaining length of the iterator.

fn chunk_puller(&self, chunk_size: usize) -> Self::ChunkPuller<'_>

Creates a ChunkPuller from the concurrent iterator. The created chunk puller can be used to pull chunk_size elements at once from the data source, rather than pulling one by one.

fn try_get_len(&self) -> Option<usize>

Returns Some(x) if the number of remaining items is known with certainly and if it is equal to x.

fn item_puller(&self) -> ItemPuller<'_, Self>

where Self: Sized,

Creates a ItemPuller from the concurrent iterator. The created item puller can be used to pull elements one by one from the data source.

fn item_puller_with_idx(&self) -> EnumeratedItemPuller<'_, Self>

where Self: Sized,

Creates a EnumeratedItemPuller from the concurrent iterator. The created item puller can be used to pull elements one by one from the data source together with the index of the elements.

fn copied<'a, T>(self) -> ConIterCopied<'a, Self, T>

where T: Send + Sync + Copy, Self: ConcurrentIter<Item = &'a T> + Sized,

Creates an iterator which copies all of its elements.

fn cloned<'a, T>(self) -> ConIterCloned<'a, Self, T>

where T: Send + Sync + Clone, Self: ConcurrentIter<Item = &'a T> + Sized,

Creates an iterator which clones all of its elements.

fn enumerate(self) -> Enumerate<Self>

where Self: Sized,

Creates an iterator which gives the current iteration count as well as the next value.

impl<I> Debug for ConIterOfIter<I>

where I: Iterator, I::Item: Send + Sync,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<I> Default for ConIterOfIter<I>

where I: Iterator + Default, I::Item: Send + Sync,

fn default() -> Self

Returns the “default value” for a type.

impl<I> ExactSizeConcurrentIter for ConIterOfIter<I>

where I: ExactSizeIterator, I::Item: Send + Sync,

fn len(&self) -> usize

Returns the number remaining elements in the concurrent iterator.

fn is_empty(&self) -> bool

Returns true if there are no elements left in the concurrent iterator. Returns false otherwise.

impl<I> Send for ConIterOfIter<I>

where I: Iterator, I::Item: Send + Sync,

impl<I> Sync for ConIterOfIter<I>

where I: Iterator, I::Item: Send + Sync,