Skip to main content

Module parallel

oxigdal_algorithms

Module parallel 

Source
Expand description

Parallel processing support for multi-core performance

This module provides parallel implementations of common geospatial operations using Rayon for work-stealing and efficient multi-threaded execution.

§Features

This module is only available when the parallel feature is enabled.

§Parallel Raster Operations

  • parallel_map_raster: Apply a function to each pixel in parallel
  • parallel_reduce_raster: Aggregate statistics from chunks
  • parallel_transform_raster: Apply transformations using thread pool

§Parallel Tile Processing

  • Process multiple tiles concurrently
  • COG pyramid generation in parallel
  • Overview computation (multiple levels simultaneously)
  • Tile compression in parallel

§Batch Processing

  • Process multiple files in parallel
  • Configurable thread pool size
  • Thread-safe result collection

§Example

use oxigdal_algorithms::parallel::*;
use oxigdal_core::buffer::RasterBuffer;
use oxigdal_core::types::RasterDataType;

// Create a raster buffer
let buffer = RasterBuffer::zeros(1000, 1000, RasterDataType::Float32);

// Apply a parallel operation
let result = parallel_map_raster(&buffer, |pixel| pixel * 2.0)?;

§Performance

Parallel operations provide 2-16x speedup on multi-core systems, depending on:

  • Number of CPU cores
  • Data size (larger datasets benefit more)
  • Operation complexity (heavier operations benefit more)
  • Memory bandwidth

§Thread Safety

All parallel operations are thread-safe and use proper synchronization. Errors are collected and returned in a thread-safe manner.

§COOLJAPAN Policy Compliance

  • Pure Rust (no C/Fortran dependencies)
  • No unwrap() or expect() in production code
  • Feature-gated (default OFF to keep dependencies minimal)
  • Comprehensive error handling

Re-exports§

pub use batch::BatchConfig;
pub use batch::BatchResult;
pub use batch::parallel_batch_process;
pub use batch::parallel_map;
pub use raster::ChunkConfig;
pub use raster::ReduceOp;
pub use raster::parallel_focal_mean;
pub use raster::parallel_focal_median;
pub use raster::parallel_map_raster;
pub use raster::parallel_map_raster_with_config;
pub use raster::parallel_reduce_raster;
pub use raster::parallel_transform_raster;
pub use raster::FocalOp;
pub use raster::focal_parallel;
pub use raster::hillshade_parallel;
pub use raster::slope_parallel;
pub use tiles::TileConfig;
pub use tiles::TileProcessor;
pub use tiles::parallel_generate_overviews;
pub use tiles::parallel_process_tiles;

Modules§

batch
Parallel batch processing
raster
Parallel raster operations
tiles
Parallel tile processing

Structs§

ParallelConfig
Configuration for parallel processing

Functions§

calculate_chunk_size
Calculates optimal chunk size based on data size and CPU count