Summary

Struct Summary 

Source
pub struct Summary { /* private fields */ }
Expand description

Single-level summary tree for task work-stealing.

This tree tracks ONLY task signals - no yield or worker state. Each leaf represents a set of task signal words.

The Summary coordinates with SignalWakers to notify partition owners when their assigned leafs become active/inactive.

§Thread Safety

This struct is designed for high-concurrency scenarios with the following guarantees:

  • All operations use atomic primitives for lock-free operation
  • Raw pointers are used for performance but are guaranteed safe by WorkerService lifetime
  • Implements Send and Sync for cross-thread usage

§Memory Layout

  • leaf_words: Atomic bitfields tracking which signals have pending tasks
  • task_reservations: Atomic bitfields for task slot reservations within each signal
  • Round-robin cursors distribute load evenly across leaves and signals
  • Partition mapping enables work-stealing between worker threads

Implementations§

Source§

impl Summary

Source

pub fn new( leaf_count: usize, signals_per_leaf: usize, wakers: &[Arc<WorkerWaker>], worker_count: &Arc<AtomicUsize>, ) -> Self

Creates a new Summary with the specified dimensions.

§Arguments
  • leaf_count - Number of leaf nodes (typically matches worker partition count)
  • signals_per_leaf - Number of task signal words per leaf (typically tasks_per_leaf / 64)
  • wakers - Slice of SignalWakers for partition owner notification
  • worker_count - Reference to WorkerService’s worker_count atomic (single source of truth)
§Safety

The wakers slice and worker_count reference must remain valid for the lifetime of this Summary. This is guaranteed when Summary is owned by WorkerService which also owns the wakers and worker_count.

§Memory Allocation

Allocates leaf_count * signals_per_leaf * 8 bytes for reservations plus overhead for leaf words and cursors.

Source

pub fn get_worker_count(&self) -> usize

Get the current worker count from WorkerService. Reads directly from the single source of truth.

§Atomic Semantics

Uses Relaxed ordering since this is only used for informational purposes and doesn’t require synchronization with other memory operations.

Source

pub fn mark_signal_active(&self, leaf_idx: usize, signal_idx: usize) -> bool

Sets the summary bit for a task signal.

§Returns

true if the leaf was empty before setting this signal (useful for work-stealing decisions)

§Atomic Semantics

Uses fetch_or with AcqRel ordering to atomically set bits and get previous state. Notifies the partition owner if new bits were added.

Source

pub fn mark_signal_inactive(&self, leaf_idx: usize, signal_idx: usize) -> bool

Clears the summary bit for a task signal.

§Returns

true if bits were successfully cleared, false if indices are invalid or bits were already cleared

§Atomic Semantics

Uses fetch_and with AcqRel ordering to atomically clear bits and get previous state. Notifies the partition owner if the leaf became empty.

Source

pub fn reserve_task_in_leaf( &self, leaf_idx: usize, signal_idx: usize, ) -> Option<u8>

Attempts to reserve a task slot within (leaf_idx, signal_idx).

§Returns

The bit index (0-63) of the reserved slot, or None if all slots are taken

§Atomic Semantics

Implements a lock-free reservation system using atomic CAS (Compare-And-Swap) loops.

  • Uses Acquire ordering for loads to see completed reservations
  • Uses AcqRel for successful CAS to make reservation visible to others
  • Uses Acquire for failed CAS to see the updated state
§Algorithm
  1. Load current reservation bitmap
  2. Use a per-signal round-robin cursor to pick a starting bit
  3. Rotate the bitmap so trailing_zeros finds the next free slot after the cursor
  4. Attempt to atomically set that bit with CAS
  5. If CAS fails (another thread reserved it), retry with updated value
  6. Continue until success or no free bits remain
Source

pub fn release_task_in_leaf( &self, leaf_idx: usize, signal_idx: usize, bit: usize, )

Clears a previously reserved task slot.

§Atomic Semantics

Uses fetch_and with SeqCst ordering to atomically clear the reservation bit. This ensures the release is visible to other threads attempting reservations.

Source

pub fn reserve_task(&self) -> Option<(usize, usize, u8)>

Convenience function: reserve the first available task slot across the arena.

§Atomic Semantics

Uses round-robin cursors with SeqCst ordering to ensure proper synchronization between threads when selecting partitions, leaves, and signals. The actual reservation uses the CAS loop in reserve_task_in_leaf which provides proper synchronization.

Source

pub fn mark_signal_inactive_if_empty( &self, leaf_idx: usize, signal_idx: usize, signal: &AtomicU64, )

Clears the summary bit when the corresponding task signal becomes empty.

§⚠️ CORRECTNESS ISSUE - TOCTOU Race Condition

This function has an unfixed race condition between checking and clearing:

Race scenario:

  1. Thread A: signal.load() sees 0
  2. Thread B: Enqueues task, signal becomes 1, calls mark_signal_active()
  3. Thread A: Calls mark_signal_inactive(), clearing the bit Thread B just set
  4. Result: signal has tasks but summary bit is cleared → lost work notification

Why this is problematic:

  • Work-stealing threads rely on summary bits to find available work
  • Clearing the bit while tasks exist makes those tasks invisible to stealers
  • The task enqueue won’t re-set the bit because it already set it in step 2

Proper fix requires one of:

  1. Caller-side synchronization ensuring signal cannot be modified during this call
  2. API change: pass mutable/exclusive access to signal to perform atomic check-and-clear
  3. Accepting false negatives: allow summary bit to remain set even when signal is empty (wastes stealer cycles but is always safe)

Current mitigation: None - callers must ensure signal stability externally.

§Atomic Semantics

Uses Acquire ordering to ensure visibility of all prior writes to the signal.

Source

pub fn leaf_count(&self) -> usize

Source

pub fn signals_per_leaf(&self) -> usize

Source

pub fn compute_partition_owner( &self, leaf_idx: usize, worker_count: usize, ) -> usize

Compute which worker owns a given leaf based on partition assignments.

This is the inverse of Worker::compute_partition(). Given a leaf index, it determines which worker is responsible for processing tasks in that leaf.

§Partition Algorithm

Uses a balanced distribution where:

  • First leaf_count % worker_count workers get (leaf_count / worker_count) + 1 leaves
  • Remaining workers get leaf_count / worker_count leaves This ensures leaves are distributed as evenly as possible.
§Arguments
  • leaf_idx - The global leaf index (0..leaf_count)
  • worker_count - Total number of active workers
§Returns

Worker ID (0..worker_count) that owns this leaf

§Example
let owner_id = summary_tree.compute_partition_owner(leaf_idx, worker_count);
let owner_waker = &service.wakers[owner_id];
owner_waker.mark_partition_leaf_active(local_idx);
Source

pub fn partition_start_for_worker( &self, worker_id: usize, worker_count: usize, ) -> usize

Compute the partition start index for a given worker.

§Arguments
  • worker_id - Worker ID (0..worker_count)
  • worker_count - Total number of active workers
§Returns

First leaf index in this worker’s partition

Source

pub fn partition_end_for_worker( &self, worker_id: usize, worker_count: usize, ) -> usize

Compute the partition end index for a given worker.

§Arguments
  • worker_id - Worker ID (0..worker_count)
  • worker_count - Total number of active workers
§Returns

One past the last leaf index in this worker’s partition (exclusive)

Source

pub fn global_to_local_leaf_idx( &self, leaf_idx: usize, worker_id: usize, worker_count: usize, ) -> Option<usize>

Convert a global leaf index to a local index within a worker’s partition.

§Arguments
  • leaf_idx - Global leaf index
  • worker_id - Worker ID
  • worker_count - Total number of workers
§Returns

Local leaf index (0..partition_size) for use with SignalWaker partition bitmap, or None if the leaf is not in this worker’s partition

Trait Implementations§

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V