[][src]Function seq_io::parallel::read_process_recordsets

pub fn read_process_recordsets<R, W, F, O, Out>(
    reader: R,
    n_workers: u32,
    queue_len: usize,
    work: W,
    func: F
) -> Out where
    R: RecordSetReader + Send,
    R::RecordSet: Default + Send,
    O: Default + Send,
    W: Send + Sync + Fn(&mut R::RecordSet, &mut O),
    F: FnOnce(ParallelDataSets<R::RecordSet, R::Err, O>) -> Out, 

This function reads record sets and processes them in parallel threads.

  • It takes a RecordSetReader, which reads data into record sets in a background thread.
  • These are then sent to n_workers worker threads, where the heavy work is done in the work closure.
  • Once ready, the record sets and work results are sent to the main thread and provided to the func closure. The won't necessarily arrive in the same order as they were read.