Struct ConIterOfIter

Source
pub struct ConIterOfIter<I>
where I: Iterator, I::Item: Send + Sync,
{ /* private fields */ }
Expand description

Concurrent iterator of a any generic type implementing a regular Iterator.

It can be created by calling iter_into_con_iter on any iterator.

This iterator has a fundamental difference from all other concurrent iterators in the following:

  • Concurrent iterators in general allow for concurrent access to different elements of the source code without blocking each other;
  • however, concurrent iterator of a generic iterator requires to serialize generation of elements which might lead pulling threads to wait each other.

This has the following implications:

  • Whenever possible, it is better to create the concurrent iterator on the concrete type rather than the generic iterator.
  • Still, the transformed concurrent iterator allows for a very convenient way to safely share the iterator among multiple threads, simply by a shared reference.
  • Furthermore, for programs where the task performed on each element of the iterator is large enough, the overhead might be considered tolerable.

§Examples

use orx_concurrent_iter::*;

let num_threads = 4;

let data: Vec<_> = (0..100).map(|x| x.to_string()).collect();

// an arbitrary iterator
let iter = data
    .into_iter()
    .filter(|x| !x.starts_with('3'))
    .map(|x| format!("{x}!"));

// converted into a concurrent iterator and shared with multiple threads
let con_iter = iter.iter_into_con_iter();

let process = |_x: String| { /* assume actual work */ };

std::thread::scope(|s| {
    for _ in 0..num_threads {
        s.spawn(|| {
            while let Some(value) = con_iter.next() {
                assert!(!value.starts_with('3') && value.ends_with('!'));
                process(value);
            }
        });
    }
});

Trait Implementations§

Source§

impl<I> ConcurrentIter for ConIterOfIter<I>
where I: Iterator, I::Item: Send + Sync,

Source§

type Item = <I as Iterator>::Item

Type of the element that the concurrent iterator yields.
Source§

type SequentialIter = I

Type of the sequential iterator that the concurrent iterator can be converted into using the into_seq_iter method.
Source§

type ChunkPuller<'i> = ChunkPullerOfIter<'i, I> where Self: 'i

Type of the chunk puller that can be created using the chunk_puller method.
Source§

fn into_seq_iter(self) -> Self::SequentialIter

Converts the concurrent iterator into its sequential regular counterpart. Note that the sequential iterator is a regular Iterator, and hence, does not have any overhead related with atomic states. Therefore, it is useful where the program decides to iterate over a single thread rather than concurrently by multiple threads. Read more
Source§

fn skip_to_end(&self)

Immediately jumps to the end of the iterator, skipping the remaining elements. Read more
Source§

fn next(&self) -> Option<Self::Item>

Returns the next element of the iterator. It returns None if there are no more elements left. Read more
Source§

fn next_with_idx(&self) -> Option<(usize, Self::Item)>

Returns the next element of the iterator together its index. It returns None if there are no more elements left. Read more
Source§

fn size_hint(&self) -> (usize, Option<usize>)

Returns the bounds on the remaining length of the iterator. Read more
Source§

fn chunk_puller(&self, chunk_size: usize) -> Self::ChunkPuller<'_>

Creates a ChunkPuller from the concurrent iterator. The created chunk puller can be used to pull chunk_size elements at once from the data source, rather than pulling one by one. Read more
Source§

fn try_get_len(&self) -> Option<usize>

Returns Some(x) if the number of remaining items is known with certainly and if it is equal to x. Read more
Source§

fn item_puller(&self) -> ItemPuller<'_, Self>
where Self: Sized,

Creates a ItemPuller from the concurrent iterator. The created item puller can be used to pull elements one by one from the data source. Read more
Source§

fn item_puller_with_idx(&self) -> EnumeratedItemPuller<'_, Self>
where Self: Sized,

Creates a EnumeratedItemPuller from the concurrent iterator. The created item puller can be used to pull elements one by one from the data source together with the index of the elements. Read more
Source§

fn copied<'a, T>(self) -> ConIterCopied<'a, Self, T>
where T: Send + Sync + Copy, Self: ConcurrentIter<Item = &'a T> + Sized,

Creates an iterator which copies all of its elements. Read more
Source§

fn cloned<'a, T>(self) -> ConIterCloned<'a, Self, T>
where T: Send + Sync + Clone, Self: ConcurrentIter<Item = &'a T> + Sized,

Creates an iterator which clones all of its elements. Read more
Source§

fn enumerate(self) -> Enumerate<Self>
where Self: Sized,

Creates an iterator which gives the current iteration count as well as the next value. Read more
Source§

impl<I> Debug for ConIterOfIter<I>
where I: Iterator, I::Item: Send + Sync,

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl<I> Default for ConIterOfIter<I>
where I: Iterator + Default, I::Item: Send + Sync,

Source§

fn default() -> Self

Returns the “default value” for a type. Read more
Source§

impl<I> ExactSizeConcurrentIter for ConIterOfIter<I>
where I: ExactSizeIterator, I::Item: Send + Sync,

Source§

fn len(&self) -> usize

Returns the number remaining elements in the concurrent iterator. Read more
Source§

fn is_empty(&self) -> bool

Returns true if there are no elements left in the concurrent iterator. Returns false otherwise. Read more
Source§

impl<I> Send for ConIterOfIter<I>
where I: Iterator, I::Item: Send + Sync,

Source§

impl<I> Sync for ConIterOfIter<I>
where I: Iterator, I::Item: Send + Sync,

Auto Trait Implementations§

§

impl<I> !Freeze for ConIterOfIter<I>

§

impl<I> !RefUnwindSafe for ConIterOfIter<I>

§

impl<I> Unpin for ConIterOfIter<I>
where I: Unpin, <I as Iterator>::Item: Unpin,

§

impl<I> UnwindSafe for ConIterOfIter<I>
where I: UnwindSafe, <I as Iterator>::Item: UnwindSafe,

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> SoM<T> for T

Source§

fn get_ref(&self) -> &T

Returns a reference to self.
Source§

fn get_mut(&mut self) -> &mut T

Returns a mutable reference to self.
Source§

impl<T> SoR<T> for T

Source§

fn get_ref(&self) -> &T

Returns a reference to self.
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.