Module parallel 

Parallel processing enhancements and workload balancing

This module provides advanced parallel processing capabilities including:

• Thread Pool: Enhanced work-stealing thread pool with adaptive scheduling
• Scheduler: Priority-based task scheduler with profiling and cost estimation
• Load Balancer: Dynamic load balancing with NUMA awareness
• Work Stealing: High-performance work-stealing implementation
• Parallel Algorithms: Optimized parallel implementations (sort, scan, reduce, map-reduce, pipeline)

Features

Work-Stealing Thread Pool

• Per-thread work-stealing deques for efficient load distribution
• Thread affinity and CPU pinning support
• Adaptive thread count based on workload
• Priority-based task scheduling

Advanced Scheduling

• Adaptive task granularity (automatic splitting of large tasks)
• Task profiling and cost estimation
• Dynamic load balancing
• Task cancellation support

NUMA Awareness

• NUMA-aware allocation and scheduling
• Work stealing across NUMA nodes
• Memory locality optimization

Parallel Algorithms

• Parallel sort (merge sort, quick sort)
• Parallel scan (prefix sum)
• Parallel reduction with custom operations
• Parallel map-reduce operations
• Parallel pipeline processing

Example

use numrs2::parallel::{ThreadPool, ParallelArrayOps, ParallelConfig};

// Create a thread pool
let pool = ThreadPool::new().expect("Failed to create thread pool");

// Submit tasks
for i in 0..10 {
    pool.submit(move || {
        println!("Task {} executing", i);
    }).expect("Failed to submit task");
}

// Wait for completion
pool.wait().expect("Failed to wait");

// Use parallel algorithms
let config = ParallelConfig::default();
let ops = ParallelArrayOps::new(config).expect("Failed to create parallel ops");

let data = vec![5, 2, 8, 1, 9];
let mut sorted = data.clone();
ops.parallel_sort(&mut sorted).expect("Failed to sort");

Re-exports

pub use load_balancer::BalancingStrategy;

pub use load_balancer::LoadBalancer;

pub use load_balancer::LoadBalancingAdvisor;

pub use load_balancer::LoadBalancingAnalysis;

pub use load_balancer::WorkloadMetrics;

pub use parallel_algorithms::parallel_prefix_sum;

pub use parallel_algorithms::parallel_scan;

pub use parallel_algorithms::ParallelArrayOps;

pub use parallel_algorithms::ParallelConfig;

pub use parallel_algorithms::ParallelFFT;

pub use parallel_algorithms::ParallelMatrixOps;

pub use parallel_algorithms::ParallelPipeline;

pub use parallel_algorithms::ParallelQuickSort;

pub use parallel_algorithms::ScanMode;

pub use parallel_allocator::ParallelAllocator;

pub use parallel_allocator::ParallelAllocatorConfig;

pub use parallel_allocator::ThreadLocalAllocator;

pub use scheduler::ParallelScheduler;

pub use scheduler::SchedulerConfig;

pub use scheduler::SchedulerStats;

pub use scheduler::TaskPriority;

pub use thread_pool::Priority;

pub use thread_pool::ThreadPool;

pub use thread_pool::ThreadPoolConfig;

pub use thread_pool::ThreadPoolStats;

pub use work_stealing::task;

pub use work_stealing::PoolStats;

pub use work_stealing::Task;

pub use work_stealing::TaskResult;

pub use work_stealing::WorkStealingConfig;

pub use work_stealing::WorkStealingPool;

Modules

load_balancer

Dynamic load balancing for parallel workloads

parallel_algorithms

Parallel algorithms optimized for numerical computations

parallel_allocator

Parallel memory allocation strategies

scheduler

Advanced parallel task scheduler with priority management

thread_pool

Enhanced thread pool with work-stealing deques and advanced features

work_stealing

Work-stealing thread pool implementation

Structs

ParallelContext

Global parallel execution context

Functions

global_parallel_context

Get the global parallel context

global_thread_count

Get the number of threads in the global thread pool

global_thread_pool

Get the global persistent thread pool

initialize_parallel_context

Initialize the global parallel context

shutdown_parallel_context

Shutdown the global parallel context