Module git_repository::parallel [−][src]
Expand description
Run computations in parallel, or not based the parallel feature toggle.
in_parallel(…)
The in_parallel(…) is the typical fan-out-fan-in mode of parallelism, with thread local storage
made available to a consume(…) function to process input. The result is sent to the Reduce running in the calling
thread to aggregate the results into a single output, which is returned by in_parallel().
Interruptions can be achieved by letting the reducers feed(…)` method fail.
It gets a boost in usability as it allows threads to borrow variables from the stack, most commonly the repository itself or the data to work on.
This mode of operation doesn’t lend itself perfectly to being wrapped for async as it appears like a single long-running
operation which runs as fast as possible, which is cancellable only by merit of stopping the input or stopping the output
aggregation.
reduce::Stepwise
The Stepwise iterator works exactly as in_parallel() except that the processing of the output produced by
consume(I, &mut State) -> O is made accessible by the Iterator trait’s next() method. As produced work is not
buffered, the owner of the iterator controls the progress made.
Getting the final output of the Reduce is achieved through the consuming Stepwise::finalize() method, which
is functionally equivalent to calling in_parallel().
In an async context this means that progress is only made each time next() is called on the iterator, while merely dropping
the iterator will wind down the computation without any result.
Maintaining Safety
In order to assure that threads don’t outlive the data they borrow because their handles are leaked, we enforce
the 'static lifetime for its inputs, making it less intuitive to use. It is, however, possible to produce
suitable input iterators as long as they can hold something on the heap.
Modules
Structs
Evaluate any iterator in their own thread.
Enums
An conditional EagerIter, which may become a just-in-time iterator running in the main thread depending on a condition.
Traits
An trait for aggregating items commonly produced in threads into a single result, without itself needing to be thread safe.
Functions
Read items from input and consume them in a single thread, producing an output to be collected by a reducer,
whose task is to aggregate these outputs into the final result returned by this function.
Run in_parallel() only if the given condition() returns true when eagerly evaluated.
Runs left and then right, one after another, returning their output when both are done.
A no-op returning the input _(desired_chunk_size, Some(thread_limit), thread_limit)_ used when the parallel` feature toggle is not set.