Skip to main content

Module scheduler

Module scheduler 

Source
Expand description

Byte-range planning and worker assignment (PRD §7, §8).

§Ownership model

A range is owned by at most one worker at a time. Ownership is the window [start + progress, end], where end lives in an AtomicU64 the scheduler may lower but never raise. A worker writes only below its own end.

Dynamic splitting (PRD §8) exploits exactly that asymmetry: to hand an idle worker part of a slow worker’s range, the scheduler lowers the victim’s end and creates a new range starting above it. Because the split point is chosen at least SPLIT_MARGIN bytes ahead of the victim’s current write position, and a single body chunk is far smaller than that margin, the victim cannot have written into the new owner’s territory even if it read the old end a moment before the split. No overlap, no locking on the hot path (PRD Invariant 2).

Structs§

Lease
A worker’s exclusive claim on part of the file.
Scheduler
Split
A split the scheduler performed, for the caller to persist.

Constants§

MAX_CHUNK
Largest range we plan up front. Bigger ranges mean coarser recovery after a crash and less room for the scheduler to rebalance.
MIN_CHUNK
Smallest range worth creating. Below this, per-request overhead and the extra connection cost more than the parallelism gains.
MIN_SPLIT_TAIL
Do not bother splitting unless both halves are worth having.
SPLIT_MARGIN
How far ahead of a victim’s write position a split point must sit. Must exceed the largest single body chunk a worker can write (tens of KiB in practice), with a wide safety factor.

Functions§

chunk_size
Chunk size for a fresh plan: one range per connection.
plan
Partition [0, total) into contiguous inclusive ranges.
plan_primed
A plan whose first range is exactly the bytes the priming probe already has in flight, so none of them are wasted and none are fetched twice.
plan_sequential
The single-range plan used when the server has no usable Range support or never told us the size (PRD §6: the fallback needs no user intervention).