pub fn par_unfold<Item, State, P, F, Fut>(
params: P,
init: State,
f: F,
) -> RecvStream<'static, Item>Expand description
Produce stream elements from a parallel asynchronous task.
This function spawns a set of parallel workers. Each worker produces and places
items to an output buffer. The worker pool size and buffer size is determined by
params.
Each worker receives a copy of initialized state init, then iteratively calls the function
f(worker_index, State) -> Fut. The Fut is a future that returns Option<(output, State)>.
The future updates the state and produces an output item.
If a worker receives a None, the worker with that worker index will halt, but it does not halt
the other workers. The output stream terminates after every worker halts.
use futures::prelude::*;
use par_stream::prelude::*;
use std::sync::{
atomic::{AtomicUsize, Ordering::*},
Arc,
};
let mut vec: Vec<_> = par_stream::par_unfold(
None,
Arc::new(AtomicUsize::new(0)),
|_, counter| async move {
let output = counter.fetch_add(1, SeqCst);
(output < 1000).then(|| (output, counter))
},
)
.collect()
.await;
vec.sort();
itertools::assert_equal(vec, 0..1000);