Skip to main content

create_pipeline

scirs2_datasets::parallel_preprocessing

Function create_pipeline 

Source 
pub fn create_pipeline<F>(
    preprocess_fn: F,
    num_workers: usize,
) -> ParallelPipeline
where
    F: Fn(&Array2<f64>) -> Result<Array2<f64>> + Send + Sync + 'static,
Expand description

Create a simple preprocessing pipeline

§Arguments

  • preprocess_fn - Function to apply to each chunk
  • num_workers - Number of worker threads (0 = auto)

§Returns

  • ParallelPipeline - The pipeline instance