Crate parallel_stream

Data parallelism library for async-std.

This library provides convenient parallel iteration of Streams. Analogous to how Rayon provides parallel iteration of Iterators. This allows processing data coming from a stream in parallel, enabling use of all system resources.

You can read about the design decisions and motivation in the “parallel streams” section of the “streams concurrency” blog post.

Differences with Rayon

Rayon is a data parallelism library built for synchronous Rust, powered by an underlying thread pool. async-std manages a thread pool as well, but the key difference with Rayon is that async-std (and futures) are optimized for latency, while Rayon is optimized for throughput.

As a rule of thumb: if you want to speed up doing heavy calculations you probably want to use Rayon. If you want to parallelize network requests consider using parallel-stream.

Examples

use parallel_stream::prelude::*;

#[async_std::main]
async fn main() {
    let v = vec![1, 2, 3, 4];
    let mut out: Vec<usize> = v
        .into_par_stream()
        .map(|n| async move { n * n })
        .collect()
        .await;
    out.sort();
    assert_eq!(out, vec![1, 4, 9, 16]);
}

Modules

prelude

The parallel stream prelude.

vec

Parallel types for Vec.

Structs

ForEach

Call a closure on each element of the stream.

FromStream

A parallel stream that was created from sequential stream.

Map

A parallel stream that maps value of another stream with a function.

Take

A stream that yields the first n items of another stream.

Traits

FromParallelStream

Conversion from a ParallelStream.

IntoParallelStream

Conversion into a ParallelStream.

ParallelStream

Parallel version of the standard Stream trait.

Functions

from_stream

Converts a stream into a parallel stream.