Expand description
Byte-range planning and worker assignment (PRD §7, §8).
§Ownership model
A range is owned by at most one worker at a time. Ownership is the window
[start + progress, end], where end lives in an AtomicU64 the scheduler
may lower but never raise. A worker writes only below its own end.
Dynamic splitting (PRD §8) exploits exactly that asymmetry: to hand an idle
worker part of a slow worker’s range, the scheduler lowers the victim’s
end and creates a new range starting above it. Because the split point is
chosen at least SPLIT_MARGIN bytes ahead of the victim’s current write
position, and a single body chunk is far smaller than that margin, the
victim cannot have written into the new owner’s territory even if it read
the old end a moment before the split. No overlap, no locking on the hot
path (PRD Invariant 2).
Structs§
- Lease
- A worker’s exclusive claim on part of the file.
- Scheduler
- Split
- A split the scheduler performed, for the caller to persist.
Constants§
- MAX_
CHUNK - Largest range we plan up front. Bigger ranges mean coarser recovery after a crash and less room for the scheduler to rebalance.
- MIN_
CHUNK - Smallest range worth creating. Below this, per-request overhead and the extra connection cost more than the parallelism gains.
- MIN_
SPLIT_ TAIL - Do not bother splitting unless both halves are worth having.
- SPLIT_
MARGIN - How far ahead of a victim’s write position a split point must sit. Must exceed the largest single body chunk a worker can write (tens of KiB in practice), with a wide safety factor.
Functions§
- chunk_
size - Chunk size for a fresh plan: one range per connection.
- plan
- Partition
[0, total)into contiguous inclusive ranges. - plan_
primed - A plan whose first range is exactly the bytes the priming probe already has in flight, so none of them are wasted and none are fetched twice.
- plan_
sequential - The single-range plan used when the server has no usable
Rangesupport or never told us the size (PRD §6: the fallback needs no user intervention).