Expand description
Crate for computing function over iterator in a streaming fashion using multi-threading while preserving order.
§Examples
let xs: &[u64] = &[100, 4, 3, 2, 1, 0, 1, 2, 3, 4, 5];
let mut ys = Vec::new();
let f = |x| x*x;
let res: Result<usize, ()> = parstream::run(
xs, 4,
// closure which is called for every x in xs
|x| {
std::thread::sleep(std::time::Duration::from_millis(*x));
Ok(f(x))
},
// closure which is called for every result with preserved order
|y| {
ys.push(y);
Ok(())
},
);
assert_eq!(res, Ok(xs.len()));
assert_eq!(ys, xs.iter().map(f).collect::<Vec<_>>());
If one of callbacks will return error, no new tasks will be started and
run
will end as soon as possible (after threads cleanup) to report this
error to caller.
#[derive(Eq, PartialEq, Debug)]
struct MyError(usize);
let xs: &[u64] = &[100, 4, 3, 2, 1, 0, 1, 2, 3, 4, 5];
let mut ys = Vec::new();
let f = |x| x*x;
let res = parstream::run(xs.iter().enumerate(), 4,
|(i, x)| {
std::thread::sleep(std::time::Duration::from_millis(*x));
if *x == 0 { return Err(MyError(i)); }
Ok(f(x))
},
|y| {
ys.push(y);
Ok(())
},
);
assert_eq!(res, Err(MyError(5)));
assert_eq!(ys.len(), 0);
§Warnings
The first closure in run
should not panic as it will lead to a deadlock!
Also report thread will not recover after the second closure panic, it will
not result in deadlock, but the second closure will not be called anymore.
Functions§
- run
- Compute
f(x)
for everyx
inxs
using thread pool and callreport
for every result and preserve order of elements.