pub struct NodeWorker { /* private fields */ }Expand description
Worker-side driver for one rank: walks the rank’s deterministic owned-chunk
sequence, turning row slices into shippable NodePartials.
The worker does not do I/O: the caller streams rows (from its shards /
object store) for the row range NodeWorker::next_chunk_rows names, hands
them to NodeWorker::emit, and ships the returned partial. The cursor
advances only on emit, so “ship durably, then checkpoint” gives exactly-
once production under crash-resume (re-shipping an already-folded chunk is
rejected by the coordinator as a duplicate, which is the safe failure).
Implementations§
Source§impl NodeWorker
impl NodeWorker
Sourcepub fn new(partition: CrossNodePartition, rank: usize) -> Result<Self, String>
pub fn new(partition: CrossNodePartition, rank: usize) -> Result<Self, String>
Fresh worker for rank, starting at the beginning of its sequence.
Sourcepub fn resume(state: NodeWorkerCheckpoint) -> Result<Self, String>
pub fn resume(state: NodeWorkerCheckpoint) -> Result<Self, String>
Resume a (replacement) worker from a serialized cursor. Validates the cursor against the partition so a checkpoint from a different pass is rejected loudly.
Sourcepub fn checkpoint(&self) -> NodeWorkerCheckpoint
pub fn checkpoint(&self) -> NodeWorkerCheckpoint
Serialize the cursor. Write this (durably) after each successful ship.
Sourcepub fn next_chunk_rows(&self) -> Option<(usize, Range<usize>)>
pub fn next_chunk_rows(&self) -> Option<(usize, Range<usize>)>
Global chunk index and row range of the next chunk to compute, or
None when done. The caller fetches exactly these rows.
Sourcepub fn emit(&mut self, rows: ArrayView2<'_, f64>) -> Result<NodePartial, String>
pub fn emit(&mut self, rows: ArrayView2<'_, f64>) -> Result<NodePartial, String>
Compute the next chunk’s deterministic partial from its rows and advance
the cursor. rows must be exactly the rows of
NodeWorker::next_chunk_rows (shape-validated here; content is the
caller’s contract, same as the in-process path).
Trait Implementations§
Source§impl Clone for NodeWorker
impl Clone for NodeWorker
Source§fn clone(&self) -> NodeWorker
fn clone(&self) -> NodeWorker
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreAuto Trait Implementations§
impl Freeze for NodeWorker
impl RefUnwindSafe for NodeWorker
impl Send for NodeWorker
impl Sync for NodeWorker
impl Unpin for NodeWorker
impl UnsafeUnpin for NodeWorker
impl UnwindSafe for NodeWorker
Blanket Implementations§
impl<T> Allocation for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> DistributionExt for Twhere
T: ?Sized,
impl<T> DistributionExt for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> Pointable for T
impl<T> Pointable for T
impl<T> Read<Exclusive, BecauseExclusive> for Twhere
T: ?Sized,
Source§impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
Source§fn to_subset(&self) -> Option<SS>
fn to_subset(&self) -> Option<SS>
self from the equivalent element of its
superset. Read moreSource§fn is_in_subset(&self) -> bool
fn is_in_subset(&self) -> bool
self is actually part of its subset T (and can be converted to it).Source§fn to_subset_unchecked(&self) -> SS
fn to_subset_unchecked(&self) -> SS
self.to_subset but without any property checks. Always succeeds.Source§fn from_subset(element: &SS) -> SP
fn from_subset(element: &SS) -> SP
self to the equivalent element of its superset.