Skip to main content

process_chunks_parallel

hexz_core::ops::parallel_pack

Function process_chunks_parallel 

Source 
pub fn process_chunks_parallel<W>(
    chunks: Vec<RawChunk>,
    compressor: Box<dyn Compressor + Send + Sync>,
    config: ParallelPackConfig,
    writer: W,
) -> Result<()>
where
    W: FnMut(CompressedChunk) -> Result<()>,
Expand description

Process chunks in parallel with multiple worker threads

§Arguments

  • chunks - Vector of raw chunks to process
  • compressor - Compressor to use (must be Send + Sync)
  • config - Parallel processing configuration
  • writer - Callback for each compressed chunk (handles dedup + write)

§Returns

Ok(()) on success, or Error if any worker fails