[−][src]Function seq_io::parallel::read_process_recordsets
pub fn read_process_recordsets<R, W, F, O, Out>(
reader: R,
n_workers: u32,
queue_len: usize,
work: W,
func: F
) -> Out where
R: RecordSetReader + Send,
R::RecordSet: Default + Send,
O: Default + Send,
W: Send + Sync + Fn(&mut R::RecordSet, &mut O),
F: FnOnce(ParallelDataSets<R::RecordSet, R::Err, O>) -> Out,
This function reads record sets and processes them in parallel threads.
- It takes a
RecordSetReader
, which reads data into record sets in a background thread. - These are then sent to
n_workers
worker threads, where the heavy work is done in thework
closure. - Once ready, the record sets and work results are sent to the main thread
and provided to the
func
closure. The won't necessarily arrive in the same order as they were read.