Trait MemoryMappedChunksParallel 

pub trait MemoryMappedChunksParallel<A: Clone + Copy + 'static + Send + Sync>: MemoryMappedChunks<A> {
    // Required methods
    fn process_chunks_parallel<F, R>(
        &self,
        strategy: ChunkingStrategy,
        f: F,
    ) -> Vec<R>
       where F: Fn(&[A], usize) -> R + Send + Sync,
             R: Send;
    fn process_chunks_mut_parallel<F>(
        &mut self,
        strategy: ChunkingStrategy,
        f: F,
    )
       where F: Fn(&mut [A], usize) + Send + Sync;
}

Extension trait for parallel processing of memory-mapped arrays.

This trait is only available when the ‘parallel’ feature is enabled. It extends the MemoryMappedChunks trait with parallel processing capabilities.

Required Methods

fn process_chunks_parallel<F, R>( &self, strategy: ChunkingStrategy, f: F, ) -> Vec<R>

where F: Fn(&[A], usize) -> R + Send + Sync, R: Send,

Process chunks in parallel and collect the results.

This method works like process_chunks but processes the chunks in parallel using Rayon. It’s useful for computationally intensive operations on large datasets.

Arguments

• strategy - The chunking strategy to determine chunk sizes
• f - A function that processes each chunk and returns a result

Returns

A vector containing the results from processing each chunk, in chunk order

Examples

use scirs2_core::memory_efficient::{create_mmap, AccessMode, ChunkingStrategy, MemoryMappedChunks, MemoryMappedChunksParallel};
use scirs2_core::ndarray::Array1;

// Create a memory-mapped array with 20 elements (small numbers to avoid overflow)
let data = Array1::<i32>::from_vec((1..=20).collect());
let file_path = "example.bin";  // In practice, use a proper temporary path
let mmap = create_mmap(&data, file_path.as_ref(), AccessMode::Write, 0).expect("Operation failed");

// Calculate the sum of each chunk in parallel
let chunk_sums = mmap.process_chunks_parallel(
    ChunkingStrategy::Fixed(5),
    |chunk, chunk_idx| chunk.iter().sum::<i32>()
);

// We should have 4 chunks with reasonable sums
assert_eq!(chunk_sums.len(), 4);

fn process_chunks_mut_parallel<F>(&mut self, strategy: ChunkingStrategy, f: F)

where F: Fn(&mut [A], usize) + Send + Sync,

Process chunks in parallel with a mutable function.

This method works like process_chunks_mut but processes the chunks in parallel using Rayon. It’s useful for computationally intensive operations on large datasets.

Arguments

• strategy - The chunking strategy to determine chunk sizes
• f - A function that processes and potentially modifies each chunk

Examples

use scirs2_core::memory_efficient::{create_mmap, AccessMode, ChunkingStrategy, MemoryMappedChunks, MemoryMappedChunksParallel};
use scirs2_core::ndarray::Array1;

// Create a memory-mapped array with 100 zeros
let data = Array1::<f64>::zeros(100);
let file_path = "example.bin";  // In practice, use a proper temporary path
let mut mmap = create_mmap(&data, file_path.as_ref(), AccessMode::Write, 0).expect("Operation failed");

// Modify each chunk in parallel: set elements to their index
mmap.process_chunks_mut_parallel(
    ChunkingStrategy::Fixed(10),
    |chunk, chunk_idx| {
        for (i, elem) in chunk.iter_mut().enumerate() {
            *elem = (chunk_idx * 10 + i) as f64;
        }
    }
);

Notes

Even when used in parallel, this method ensures that file writes are safe and do not conflict with each other by collecting all modifications and applying them sequentially.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors

impl<A: Clone + Copy + 'static + Send + Sync> MemoryMappedChunksParallel<A> for MemoryMappedArray<A>

Available on crate feature parallel only.