Struct TableParIterator

pub struct TableParIterator<'a, T> {
    pub table: &'a MetadataTable<'a, T>,
    pub range: Range<u32>,
}

Parallel iterator for metadata table rows.

This iterator enables concurrent processing of table rows across multiple threads using the Rayon parallel processing framework. It automatically distributes work and handles synchronization, providing significant performance improvements for CPU-intensive operations on large tables.

Features

• Automatic parallelization: Work is distributed across available CPU cores
• Load balancing: Dynamic work stealing ensures optimal CPU utilization
• Error handling: Built-in support for early termination on errors
• Type safety: Compile-time guarantees about thread safety requirements

Requirements

The row type T must implement Send + Sync to enable safe parallel processing. This ensures that rows can be safely transferred between threads and accessed concurrently.

Usage

Created through MetadataTable::par_iter() and supports all Rayon parallel iterator operations

Fields

table: &'a MetadataTable<'a, T>

Reference to the table being iterated

range: Range<u32>

Range of row indices to process

Implementations

impl<'a, T: RowReadable + Send + Sync + 'a> TableParIterator<'a, T>

pub fn try_for_each<F>(self, op: F) -> Result<()>

where F: Fn(T) -> Result<()> + Send + Sync,

Processes the iterator in parallel with early error detection and termination.

This method provides a parallel equivalent to the standard iterator’s try_for_each, executing the provided operation on each row concurrently while monitoring for errors. If any operation fails, processing stops and the first error encountered is returned.

Arguments

• op - A closure that takes each row and returns a Result. Must be Send + Sync to enable safe parallel execution.

Returns

Returns Ok(()) if all operations complete successfully, or the first error encountered during parallel processing.

Panics

This function will panic if the mutex is poisoned during error handling.

Errors

Returns an error if any operation applied to an item returns an error. The first error encountered is returned.

Trait Implementations

impl<T: RowReadable + Send + Sync> IndexedParallelIterator for TableParIterator<'_, T>

fn len(&self) -> usize

Produces an exact count of how many items this iterator will produce, presuming no panic occurs.

fn drive<C>(self, consumer: C) -> C::Result

where C: Consumer<Self::Item>,

Internal method used to define the behavior of this parallel iterator. You should not need to call this directly.

fn with_producer<CB>(self, callback: CB) -> CB::Output

where CB: ProducerCallback<Self::Item>,

Internal method used to define the behavior of this parallel iterator. You should not need to call this directly.

fn by_exponential_blocks(self) -> ExponentialBlocks<Self>

Divides an iterator into sequential blocks of exponentially-increasing size.

fn by_uniform_blocks(self, block_size: usize) -> UniformBlocks<Self>

Divides an iterator into sequential blocks of the given size.

fn collect_into_vec(self, target: &mut Vec<Self::Item>)

Collects the results of the iterator into the specified vector. The vector is always cleared before execution begins. If possible, reusing the vector across calls can lead to better performance since it reuses the same backing buffer.

fn unzip_into_vecs<A, B>(self, left: &mut Vec<A>, right: &mut Vec<B>)

where Self: IndexedParallelIterator<Item = (A, B)>, A: Send, B: Send,

Unzips the results of the iterator into the specified vectors. The vectors are always cleared before execution begins. If possible, reusing the vectors across calls can lead to better performance since they reuse the same backing buffer.

fn zip<Z>(self, zip_op: Z) -> Zip<Self, <Z as IntoParallelIterator>::Iter>

where Z: IntoParallelIterator, <Z as IntoParallelIterator>::Iter: IndexedParallelIterator,

Iterates over tuples (A, B), where the items A are from this iterator and B are from the iterator given as argument. Like the zip method on ordinary iterators, if the two iterators are of unequal length, you only get the items they have in common.

fn zip_eq<Z>(self, zip_op: Z) -> ZipEq<Self, <Z as IntoParallelIterator>::Iter>

where Z: IntoParallelIterator, <Z as IntoParallelIterator>::Iter: IndexedParallelIterator,

The same as Zip, but requires that both iterators have the same length.

fn interleave<I>( self, other: I, ) -> Interleave<Self, <I as IntoParallelIterator>::Iter>

where I: IntoParallelIterator<Item = Self::Item>, <I as IntoParallelIterator>::Iter: IndexedParallelIterator,

Interleaves elements of this iterator and the other given iterator. Alternately yields elements from this iterator and the given iterator, until both are exhausted. If one iterator is exhausted before the other, the last elements are provided from the other.

fn interleave_shortest<I>( self, other: I, ) -> InterleaveShortest<Self, <I as IntoParallelIterator>::Iter>

where I: IntoParallelIterator<Item = Self::Item>, <I as IntoParallelIterator>::Iter: IndexedParallelIterator,

Interleaves elements of this iterator and the other given iterator, until one is exhausted.

fn chunks(self, chunk_size: usize) -> Chunks<Self>

Splits an iterator up into fixed-size chunks.

fn fold_chunks<T, ID, F>( self, chunk_size: usize, identity: ID, fold_op: F, ) -> FoldChunks<Self, ID, F>

where ID: Fn() -> T + Send + Sync, F: Fn(T, Self::Item) -> T + Send + Sync, T: Send,

Splits an iterator into fixed-size chunks, performing a sequential fold() on each chunk.

fn fold_chunks_with<T, F>( self, chunk_size: usize, init: T, fold_op: F, ) -> FoldChunksWith<Self, T, F>

where T: Send + Clone, F: Fn(T, Self::Item) -> T + Send + Sync,

Splits an iterator into fixed-size chunks, performing a sequential fold() on each chunk.

fn cmp<I>(self, other: I) -> Ordering

where I: IntoParallelIterator<Item = Self::Item>, <I as IntoParallelIterator>::Iter: IndexedParallelIterator, Self::Item: Ord,

Lexicographically compares the elements of this ParallelIterator with those of another.

fn partial_cmp<I>(self, other: I) -> Option<Ordering>

where I: IntoParallelIterator, <I as IntoParallelIterator>::Iter: IndexedParallelIterator, Self::Item: PartialOrd<<I as IntoParallelIterator>::Item>,

Lexicographically compares the elements of this ParallelIterator with those of another.

fn eq<I>(self, other: I) -> bool

where I: IntoParallelIterator, <I as IntoParallelIterator>::Iter: IndexedParallelIterator, Self::Item: PartialEq<<I as IntoParallelIterator>::Item>,

Determines if the elements of this ParallelIterator are equal to those of another

fn ne<I>(self, other: I) -> bool

where I: IntoParallelIterator, <I as IntoParallelIterator>::Iter: IndexedParallelIterator, Self::Item: PartialEq<<I as IntoParallelIterator>::Item>,

Determines if the elements of this ParallelIterator are unequal to those of another

fn lt<I>(self, other: I) -> bool

where I: IntoParallelIterator, <I as IntoParallelIterator>::Iter: IndexedParallelIterator, Self::Item: PartialOrd<<I as IntoParallelIterator>::Item>,

Determines if the elements of this ParallelIterator are lexicographically less than those of another.

fn le<I>(self, other: I) -> bool

where I: IntoParallelIterator, <I as IntoParallelIterator>::Iter: IndexedParallelIterator, Self::Item: PartialOrd<<I as IntoParallelIterator>::Item>,

Determines if the elements of this ParallelIterator are less than or equal to those of another.

fn gt<I>(self, other: I) -> bool

where I: IntoParallelIterator, <I as IntoParallelIterator>::Iter: IndexedParallelIterator, Self::Item: PartialOrd<<I as IntoParallelIterator>::Item>,

Determines if the elements of this ParallelIterator are lexicographically greater than those of another.

fn ge<I>(self, other: I) -> bool

where I: IntoParallelIterator, <I as IntoParallelIterator>::Iter: IndexedParallelIterator, Self::Item: PartialOrd<<I as IntoParallelIterator>::Item>,

Determines if the elements of this ParallelIterator are greater than or equal to those of another.

fn enumerate(self) -> Enumerate<Self>

Yields an index along with each item.

fn step_by(self, step: usize) -> StepBy<Self>

Creates an iterator that steps by the given amount

fn skip(self, n: usize) -> Skip<Self>

Creates an iterator that skips the first n elements.

fn take(self, n: usize) -> Take<Self>

Creates an iterator that yields the first n elements.

fn position_any<P>(self, predicate: P) -> Option<usize>

where P: Fn(Self::Item) -> bool + Sync + Send,

Searches for some item in the parallel iterator that matches the given predicate, and returns its index. Like ParallelIterator::find_any, the parallel search will not necessarily find the first match, and once a match is found we’ll attempt to stop processing any more.

fn position_first<P>(self, predicate: P) -> Option<usize>

where P: Fn(Self::Item) -> bool + Sync + Send,

Searches for the sequentially first item in the parallel iterator that matches the given predicate, and returns its index.

fn position_last<P>(self, predicate: P) -> Option<usize>

where P: Fn(Self::Item) -> bool + Sync + Send,

Searches for the sequentially last item in the parallel iterator that matches the given predicate, and returns its index.

fn positions<P>(self, predicate: P) -> Positions<Self, P>

where P: Fn(Self::Item) -> bool + Sync + Send,

Searches for items in the parallel iterator that match the given predicate, and returns their indices.

fn rev(self) -> Rev<Self>

Produces a new iterator with the elements of this iterator in reverse order.

fn with_min_len(self, min: usize) -> MinLen<Self>

Sets the minimum length of iterators desired to process in each rayon job. Rayon will not split any smaller than this length, but of course an iterator could already be smaller to begin with.

fn with_max_len(self, max: usize) -> MaxLen<Self>

Sets the maximum length of iterators desired to process in each rayon job. Rayon will try to split at least below this length, unless that would put it below the length from with_min_len(). For example, given min=10 and max=15, a length of 16 will not be split any further.

impl<T: RowReadable + Send + Sync> ParallelIterator for TableParIterator<'_, T>

type Item = T

The type of item that this parallel iterator produces. For example, if you use the for_each method, this is the type of item that your closure will be invoked with.

fn drive_unindexed<C>(self, consumer: C) -> C::Result

where C: UnindexedConsumer<Self::Item>,

Internal method used to define the behavior of this parallel iterator. You should not need to call this directly.

fn for_each<OP>(self, op: OP)

where OP: Fn(Self::Item) + Sync + Send,

Executes OP on each item produced by the iterator, in parallel.

fn for_each_with<OP, T>(self, init: T, op: OP)

where OP: Fn(&mut T, Self::Item) + Sync + Send, T: Send + Clone,

Executes OP on the given init value with each item produced by the iterator, in parallel.

fn for_each_init<OP, INIT, T>(self, init: INIT, op: OP)

where OP: Fn(&mut T, Self::Item) + Sync + Send, INIT: Fn() -> T + Sync + Send,

Executes OP on a value returned by init with each item produced by the iterator, in parallel.

fn try_for_each<OP, R>(self, op: OP) -> R

where OP: Fn(Self::Item) -> R + Sync + Send, R: Try<Output = ()> + Send,

Executes a fallible OP on each item produced by the iterator, in parallel.

fn try_for_each_with<OP, T, R>(self, init: T, op: OP) -> R

where OP: Fn(&mut T, Self::Item) -> R + Sync + Send, T: Send + Clone, R: Try<Output = ()> + Send,

Executes a fallible OP on the given init value with each item produced by the iterator, in parallel.

fn try_for_each_init<OP, INIT, T, R>(self, init: INIT, op: OP) -> R

where OP: Fn(&mut T, Self::Item) -> R + Sync + Send, INIT: Fn() -> T + Sync + Send, R: Try<Output = ()> + Send,

Executes a fallible OP on a value returned by init with each item produced by the iterator, in parallel.

fn count(self) -> usize

Counts the number of items in this parallel iterator.

fn map<F, R>(self, map_op: F) -> Map<Self, F>

where F: Fn(Self::Item) -> R + Sync + Send, R: Send,

Applies map_op to each item of this iterator, producing a new iterator with the results.

fn map_with<F, T, R>(self, init: T, map_op: F) -> MapWith<Self, T, F>

where F: Fn(&mut T, Self::Item) -> R + Sync + Send, T: Send + Clone, R: Send,

Applies map_op to the given init value with each item of this iterator, producing a new iterator with the results.

fn map_init<F, INIT, T, R>( self, init: INIT, map_op: F, ) -> MapInit<Self, INIT, F>

where F: Fn(&mut T, Self::Item) -> R + Sync + Send, INIT: Fn() -> T + Sync + Send, R: Send,

Applies map_op to a value returned by init with each item of this iterator, producing a new iterator with the results.

fn cloned<'a, T>(self) -> Cloned<Self>

where T: 'a + Clone + Send, Self: ParallelIterator<Item = &'a T>,

Creates an iterator which clones all of its elements. This may be useful when you have an iterator over &T, but you need T, and that type implements Clone. See also copied().

fn copied<'a, T>(self) -> Copied<Self>

where T: 'a + Copy + Send, Self: ParallelIterator<Item = &'a T>,

Creates an iterator which copies all of its elements. This may be useful when you have an iterator over &T, but you need T, and that type implements Copy. See also cloned().

fn inspect<OP>(self, inspect_op: OP) -> Inspect<Self, OP>

where OP: Fn(&Self::Item) + Sync + Send,

Applies inspect_op to a reference to each item of this iterator, producing a new iterator passing through the original items. This is often useful for debugging to see what’s happening in iterator stages.

fn update<F>(self, update_op: F) -> Update<Self, F>

where F: Fn(&mut Self::Item) + Sync + Send,

Mutates each item of this iterator before yielding it.

fn filter<P>(self, filter_op: P) -> Filter<Self, P>

where P: Fn(&Self::Item) -> bool + Sync + Send,

Applies filter_op to each item of this iterator, producing a new iterator with only the items that gave true results.

fn filter_map<P, R>(self, filter_op: P) -> FilterMap<Self, P>

where P: Fn(Self::Item) -> Option<R> + Sync + Send, R: Send,

Applies filter_op to each item of this iterator to get an Option, producing a new iterator with only the items from Some results.

fn flat_map<F, PI>(self, map_op: F) -> FlatMap<Self, F>

where F: Fn(Self::Item) -> PI + Sync + Send, PI: IntoParallelIterator,

Applies map_op to each item of this iterator to get nested parallel iterators, producing a new parallel iterator that flattens these back into one.

fn flat_map_iter<F, SI>(self, map_op: F) -> FlatMapIter<Self, F>

where F: Fn(Self::Item) -> SI + Sync + Send, SI: IntoIterator, <SI as IntoIterator>::Item: Send,

Applies map_op to each item of this iterator to get nested serial iterators, producing a new parallel iterator that flattens these back into one.

fn flatten(self) -> Flatten<Self>

where Self::Item: IntoParallelIterator,

An adaptor that flattens parallel-iterable Items into one large iterator.

fn flatten_iter(self) -> FlattenIter<Self>

where Self::Item: IntoIterator, <Self::Item as IntoIterator>::Item: Send,

An adaptor that flattens serial-iterable Items into one large iterator.

fn reduce<OP, ID>(self, identity: ID, op: OP) -> Self::Item

where OP: Fn(Self::Item, Self::Item) -> Self::Item + Sync + Send, ID: Fn() -> Self::Item + Sync + Send,

Reduces the items in the iterator into one item using op. The argument identity should be a closure that can produce “identity” value which may be inserted into the sequence as needed to create opportunities for parallel execution. So, for example, if you are doing a summation, then identity() ought to produce something that represents the zero for your type (but consider just calling sum() in that case).

fn reduce_with<OP>(self, op: OP) -> Option<Self::Item>

where OP: Fn(Self::Item, Self::Item) -> Self::Item + Sync + Send,

Reduces the items in the iterator into one item using op. If the iterator is empty, None is returned; otherwise, Some is returned.

fn try_reduce<T, OP, ID>(self, identity: ID, op: OP) -> Self::Item

where OP: Fn(T, T) -> Self::Item + Sync + Send, ID: Fn() -> T + Sync + Send, Self::Item: Try<Output = T>,

Reduces the items in the iterator into one item using a fallible op. The identity argument is used the same way as in reduce().

fn try_reduce_with<T, OP>(self, op: OP) -> Option<Self::Item>

where OP: Fn(T, T) -> Self::Item + Sync + Send, Self::Item: Try<Output = T>,

Reduces the items in the iterator into one item using a fallible op.

fn fold<T, ID, F>(self, identity: ID, fold_op: F) -> Fold<Self, ID, F>

where F: Fn(T, Self::Item) -> T + Sync + Send, ID: Fn() -> T + Sync + Send, T: Send,

Parallel fold is similar to sequential fold except that the sequence of items may be subdivided before it is folded. Consider a list of numbers like 22 3 77 89 46. If you used sequential fold to add them (fold(0, |a,b| a+b), you would wind up first adding 0 + 22, then 22 + 3, then 25 + 77, and so forth. The parallel fold works similarly except that it first breaks up your list into sublists, and hence instead of yielding up a single sum at the end, it yields up multiple sums. The number of results is nondeterministic, as is the point where the breaks occur.

fn fold_with<F, T>(self, init: T, fold_op: F) -> FoldWith<Self, T, F>

where F: Fn(T, Self::Item) -> T + Sync + Send, T: Send + Clone,

Applies fold_op to the given init value with each item of this iterator, finally producing the value for further use.

fn try_fold<T, R, ID, F>( self, identity: ID, fold_op: F, ) -> TryFold<Self, R, ID, F>

where F: Fn(T, Self::Item) -> R + Sync + Send, ID: Fn() -> T + Sync + Send, R: Try<Output = T> + Send,

Performs a fallible parallel fold.

fn try_fold_with<F, T, R>(self, init: T, fold_op: F) -> TryFoldWith<Self, R, F>

where F: Fn(T, Self::Item) -> R + Sync + Send, R: Try<Output = T> + Send, T: Clone + Send,

Performs a fallible parallel fold with a cloneable init value.

fn sum<S>(self) -> S

where S: Send + Sum<Self::Item> + Sum,

Sums up the items in the iterator.

fn product<P>(self) -> P

where P: Send + Product<Self::Item> + Product,

Multiplies all the items in the iterator.

fn min(self) -> Option<Self::Item>

where Self::Item: Ord,

Computes the minimum of all the items in the iterator. If the iterator is empty, None is returned; otherwise, Some(min) is returned.

fn min_by<F>(self, f: F) -> Option<Self::Item>

where F: Sync + Send + Fn(&Self::Item, &Self::Item) -> Ordering,

Computes the minimum of all the items in the iterator with respect to the given comparison function. If the iterator is empty, None is returned; otherwise, Some(min) is returned.

fn min_by_key<K, F>(self, f: F) -> Option<Self::Item>

where K: Ord + Send, F: Sync + Send + Fn(&Self::Item) -> K,

Computes the item that yields the minimum value for the given function. If the iterator is empty, None is returned; otherwise, Some(item) is returned.

fn max(self) -> Option<Self::Item>

where Self::Item: Ord,

Computes the maximum of all the items in the iterator. If the iterator is empty, None is returned; otherwise, Some(max) is returned.

fn max_by<F>(self, f: F) -> Option<Self::Item>

where F: Sync + Send + Fn(&Self::Item, &Self::Item) -> Ordering,

Computes the maximum of all the items in the iterator with respect to the given comparison function. If the iterator is empty, None is returned; otherwise, Some(max) is returned.

fn max_by_key<K, F>(self, f: F) -> Option<Self::Item>

where K: Ord + Send, F: Sync + Send + Fn(&Self::Item) -> K,

Computes the item that yields the maximum value for the given function. If the iterator is empty, None is returned; otherwise, Some(item) is returned.

fn chain<C>(self, chain: C) -> Chain<Self, <C as IntoParallelIterator>::Iter>

where C: IntoParallelIterator<Item = Self::Item>,

Takes two iterators and creates a new iterator over both.

fn find_any<P>(self, predicate: P) -> Option<Self::Item>

where P: Fn(&Self::Item) -> bool + Sync + Send,

Searches for some item in the parallel iterator that matches the given predicate and returns it. This operation is similar to find on sequential iterators but the item returned may not be the first one in the parallel sequence which matches, since we search the entire sequence in parallel.

fn find_first<P>(self, predicate: P) -> Option<Self::Item>

where P: Fn(&Self::Item) -> bool + Sync + Send,

Searches for the sequentially first item in the parallel iterator that matches the given predicate and returns it.

fn find_last<P>(self, predicate: P) -> Option<Self::Item>

where P: Fn(&Self::Item) -> bool + Sync + Send,

Searches for the sequentially last item in the parallel iterator that matches the given predicate and returns it.

fn find_map_any<P, R>(self, predicate: P) -> Option<R>

where P: Fn(Self::Item) -> Option<R> + Sync + Send, R: Send,

Applies the given predicate to the items in the parallel iterator and returns any non-None result of the map operation.

fn find_map_first<P, R>(self, predicate: P) -> Option<R>

where P: Fn(Self::Item) -> Option<R> + Sync + Send, R: Send,

Applies the given predicate to the items in the parallel iterator and returns the sequentially first non-None result of the map operation.

fn find_map_last<P, R>(self, predicate: P) -> Option<R>

where P: Fn(Self::Item) -> Option<R> + Sync + Send, R: Send,

Applies the given predicate to the items in the parallel iterator and returns the sequentially last non-None result of the map operation.

fn any<P>(self, predicate: P) -> bool

where P: Fn(Self::Item) -> bool + Sync + Send,

Searches for some item in the parallel iterator that matches the given predicate, and if so returns true. Once a match is found, we’ll attempt to stop process the rest of the items. Proving that there’s no match, returning false, does require visiting every item.

fn all<P>(self, predicate: P) -> bool

where P: Fn(Self::Item) -> bool + Sync + Send,

Tests that every item in the parallel iterator matches the given predicate, and if so returns true. If a counter-example is found, we’ll attempt to stop processing more items, then return false.

fn while_some<T>(self) -> WhileSome<Self>

where Self: ParallelIterator<Item = Option<T>>, T: Send,

Creates an iterator over the Some items of this iterator, halting as soon as any None is found.

fn panic_fuse(self) -> PanicFuse<Self>

Wraps an iterator with a fuse in case of panics, to halt all threads as soon as possible.

fn collect<C>(self) -> C

where C: FromParallelIterator<Self::Item>,

Creates a fresh collection containing all the elements produced by this parallel iterator.

fn unzip<A, B, FromA, FromB>(self) -> (FromA, FromB)

where Self: ParallelIterator<Item = (A, B)>, FromA: Default + Send + ParallelExtend<A>, FromB: Default + Send + ParallelExtend<B>, A: Send, B: Send,

Unzips the items of a parallel iterator into a pair of arbitrary ParallelExtend containers.

fn partition<A, B, P>(self, predicate: P) -> (A, B)

where A: Default + Send + ParallelExtend<Self::Item>, B: Default + Send + ParallelExtend<Self::Item>, P: Fn(&Self::Item) -> bool + Sync + Send,

Partitions the items of a parallel iterator into a pair of arbitrary ParallelExtend containers. Items for which the predicate returns true go into the first container, and the rest go into the second.

fn partition_map<A, B, P, L, R>(self, predicate: P) -> (A, B)

where A: Default + Send + ParallelExtend<L>, B: Default + Send + ParallelExtend<R>, P: Fn(Self::Item) -> Either<L, R> + Sync + Send, L: Send, R: Send,

Partitions and maps the items of a parallel iterator into a pair of arbitrary ParallelExtend containers. Either::Left items go into the first container, and Either::Right items go into the second.

fn intersperse(self, element: Self::Item) -> Intersperse<Self>

where Self::Item: Clone,

Intersperses clones of an element between items of this iterator.

fn take_any(self, n: usize) -> TakeAny<Self>

Creates an iterator that yields n elements from anywhere in the original iterator.

fn skip_any(self, n: usize) -> SkipAny<Self>

Creates an iterator that skips n elements from anywhere in the original iterator.

fn take_any_while<P>(self, predicate: P) -> TakeAnyWhile<Self, P>

where P: Fn(&Self::Item) -> bool + Sync + Send,

Creates an iterator that takes elements from anywhere in the original iterator until the given predicate returns false.

fn skip_any_while<P>(self, predicate: P) -> SkipAnyWhile<Self, P>

where P: Fn(&Self::Item) -> bool + Sync + Send,

Creates an iterator that skips elements from anywhere in the original iterator until the given predicate returns false.

fn collect_vec_list(self) -> LinkedList<Vec<Self::Item>>

Collects this iterator into a linked list of vectors.

fn opt_len(&self) -> Option<usize>

Internal method used to define the behavior of this parallel iterator. You should not need to call this directly.