macro_rules! pipeline { ($s1:expr $(, $tail:expr)*) => { ... }; }
Expand description
Macro that allows to create a concurrent pipeline.
The macro takes as input a list of stages. Each stage is a node of the pipeline.
The first stage is the source of the pipeline. It can be any struct that implements the
node::Out
trait. The last stage is the sink of the pipeline. It can be any struct that
implements the node::In
trait. The stages in the middle are the processing stages of the
pipeline. They can be any struct that implements the node::InOut
trait.
The stages in the middle, the ones that implement node::InOut
trait, can be replicated.
Thanks to this it is possible to introduce parallelism in a stage of the pipeline.
When a stage is replicated, the inputs arriving at that stage are distributed equally
among all replicas. This replication mechanism enables the expression of a task farm pattern,
which in turn introduces parallelism within a stage. This parallelism allows for more efficient
and concurrent processing of data within the pipeline.
The number of replicas of a stage is defined by the
node::InOut::number_of_replicas
method of the node::InOut
trait.
The macro returns a Pipeline
struct that can be used to start and wait for the pipeline.
§Example
use ppl::prelude::*;
use ppl::templates::misc::*;
let mut pipeline = pipeline![
SourceIter::build(0..10),
Sequential::build(|el: usize| -> usize { el * 2 }),
SinkVec::build()
];
pipeline.start();
let res = pipeline.wait_end().unwrap();
assert_eq!(res, vec![0, 2, 4, 6, 8, 10, 12, 14, 16, 18]);