generate_in_chunks

Function generate_in_chunks 

Source
pub async fn generate_in_chunks<G, I, S>(
    sink: S,
    sources: I,
    num_threads: usize,
) -> Result<(), Error>
where G: Source + 'static, I: Iterator<Item = G>, S: Sink + 'static,
Expand description

Generates data in parallel from a series of Source and writes to a Sink

Each Source is a data generator that generates data directly into an in memory buffer.

This function will run the Sourcees in parallel up to num_threads. Data is written to the Sink in the order of the Sourcees in the input iterator.

G: Generator I: Iterator<Item = G> S: Sink that writes buffers somewhere