Skip to main content

Module rayon

Module rayon 

Source
Expand description

A work-stealing pool of worker threads.

A ThreadPool owns a fixed set of worker threads. Each worker keeps a deque of jobs that it pushes and pops at one end while idle workers steal from the other, and a pool-wide injector takes jobs submitted from threads outside the pool. A worker with nothing to run searches for work for a while and then sleeps. Publishing a job or setting a latch wakes sleeping workers as needed. A worker waiting for a job’s latch keeps running its pool’s jobs until it is set.

Internal join calls run two closures potentially in parallel. ThreadPool::spawn runs a detached task, and ThreadPool::broadcast and ThreadPool::spawn_broadcast run a task on every worker. The bridge maps, folds, and collects vectors by splitting them adaptively. Sorting uses par_mergesort for stable sorts and par_quicksort for unstable sorts.

§Changes vs. Upstream

This module vendors the scheduler of rayon-core 1.13.0 and the producer/consumer bridge and slice sorts of rayon 1.12.0. The upstream LICENSE-APACHE, LICENSE-MIT, and sleep protocol README.md accompany the code.

  • Moved rayon-core into this module and rewrote crate paths. Flattened the broadcast, join, spawn, and thread_pool modules beside their test directories.
  • Exports ThreadPool, ThreadPoolBuilder, ThreadPoolBuildError, and Yield through commonware_parallel::rayon. Retained free functions are crate-private or test-only.
  • Removed the global pool, build_global, initialize, Registry::current, the primed latch, and GlobalPoolAlreadyInitialized. Internal join and current_num_threads require a calling worker.
  • Removed free spawn, broadcast, and spawn_broadcast.
  • Removed scopes and their CountLatch operations, FIFO scheduling, yield_local, current_thread_has_pending_tasks, max_num_threads, and ThreadPool::join.
  • Removed Configuration, ThreadPool::new, and the deprecated error description override.
  • Removed builder options spawn_handler, build_scoped, exit_handler, panic_handler, and breadth_first. The registry starts threads directly. Uncaught detached-job and start-handler panics abort.
  • Removed the fallback for targets without threads, ThreadPoolBuildError::is_unsupported, web_spin_lock, wasm_sync, the build script, and the links key.
  • bridge.rs directly composes the producer and consumers. Removed iterator traits and wrappers, ProducerCallback, producer-side map adapters, unindexed plumbing, ranged draining, and NoopConsumer. Whole-vector draining uses inherent methods and IntoIterator. Fallible folds and reductions use Result in place of Try and ControlFlow.
  • Merge-sort chunks run through recursive join calls in the same indexed split order as the upstream par_chunks_mut iterator.
  • Failed collections drop partial outputs before returning the error, without assembling a final vector.
  • Added ThreadPool::block_on_worker to poll futures while servicing pool jobs and waiting through the scheduler’s latch and sleep protocol.
  • Added ThreadPoolBuilder::unstarted_workers for test mocks and a test-only notification when a worker commits to sleeping.
  • Tests use explicit pools and commonware_utils::test_rng. Removed compile-fail tests and tests of omitted APIs. Added worker, collection, sorting, and wakeup regressions.
  • Adapted unsafe blocks, borrowing, and syntax to Rust 2024 and workspace lints. Added SAFETY comments and local lint allowances for upstream std::sync primitives. List collection uses Vec::with_capacity in place of new followed by reserve.
  • Adapted docs to the retained API, clarified queued join execution, removed inaccessible doctests and section banners, and named the scheduler in its abort message.

Structs§

ThreadPool
Represents a user-created thread pool.
ThreadPoolBuildError
Error when initializing a thread pool.
ThreadPoolBuilder
Used to create a new ThreadPool.

Enums§

Yield
Result of a worker’s cooperative yield to its thread pool.