Macro ppl::pipeline

macro_rules! pipeline {
    ($s1:expr $(, $tail:expr)*) => { ... };
}

Macro that allows to create a concurrent pipeline.

The macro takes as input a list of stages. Each stage is a node of the pipeline.

The first stage is the source of the pipeline. It can be any struct that implements the node::Out trait. The last stage is the sink of the pipeline. It can be any struct that implements the node::In trait. The stages in the middle are the processing stages of the pipeline. They can be any struct that implements the node::InOut trait.

The stages in the middle, the ones that implement node::InOut trait, can be replicated. Thanks to this it is possible to introduce parallelism in a stage of the pipeline. When a stage is replicated, the inputs arriving at that stage are distributed equally among all replicas. This replication mechanism enables the expression of a task farm pattern, which in turn introduces parallelism within a stage. This parallelism allows for more efficient and concurrent processing of data within the pipeline. The number of replicas of a stage is defined by the node::InOut::number_of_replicas method of the node::InOut trait.

The macro returns a Pipeline struct that can be used to start and wait for the pipeline.

Example

use ppl::prelude::*;
use ppl::templates::misc::*;

let mut pipeline = pipeline![
     SourceIter::build(0..10),
     Sequential::build(|el: usize| -> usize { el * 2 }),
     SinkVec::build()
];
pipeline.start();
let res = pipeline.wait_end().unwrap();

assert_eq!(res, vec![0, 2, 4, 6, 8, 10, 12, 14, 16, 18]);