pub struct CrossNodeGramReduction { /* private fields */ }Expand description
Coordinator-side reduction: receives NodePartials from the fleet and
folds them into the single global fixed-tree accumulator.
Receipt validation is per rank and in-sequence: rank r’s i-th accepted
partial must be its i-th owned chunk. This makes the per-rank cursor in
CrossNodeCheckpoint::received_per_rank a complete description of what
has been received, which is what lets a dead rank resume from a bare
ordinal. Cross-rank arrival order is unconstrained (the inner accumulator
buffers out-of-order chunks), so slow nodes never block fast ones.
Implementations§
Source§impl CrossNodeGramReduction
impl CrossNodeGramReduction
Sourcepub fn new(partition: CrossNodePartition) -> Result<Self, String>
pub fn new(partition: CrossNodePartition) -> Result<Self, String>
Fresh coordinator for the given partition.
Sourcepub fn partition(&self) -> CrossNodePartition
pub fn partition(&self) -> CrossNodePartition
The shared partition (workers must be constructed with an equal one).
Sourcepub fn rank_cursor(&self, rank: usize) -> Option<usize>
pub fn rank_cursor(&self, rank: usize) -> Option<usize>
How many partials rank rank has had accepted — the ordinal a
replacement worker for that rank should resume from.
Sourcepub fn is_complete(&self) -> bool
pub fn is_complete(&self) -> bool
true once every chunk of every rank has been received and folded.
Sourcepub fn receive(&mut self, partial: NodePartial) -> Result<(), String>
pub fn receive(&mut self, partial: NodePartial) -> Result<(), String>
Receive one shipped partial. Validates rank, ownership, and per-rank
sequence position, then folds through
StreamingBorderGram::submit_chunk_gram (which re-validates index
range, duplicates, and partial shape). A duplicate of an already-folded
chunk — the signature of an at-least-once transport retry or a worker
that resumed from a stale cursor — is rejected with an error naming the
chunk, never silently double-counted.
Sourcepub fn checkpoint(&self) -> CrossNodeCheckpoint
pub fn checkpoint(&self) -> CrossNodeCheckpoint
Serialize the full coordinator state. Resume-equals-straight-through is inherited bit-for-bit from the inner accumulator; the per-rank cursors resume receipt validation exactly where it stopped.
Sourcepub fn resume(state: CrossNodeCheckpoint) -> Result<Self, String>
pub fn resume(state: CrossNodeCheckpoint) -> Result<Self, String>
Reconstruct a coordinator from a checkpoint, validating the cursor structure against the partition so corruption is rejected loudly.
Sourcepub fn finish(self) -> Result<Array2<f64>, String>
pub fn finish(self) -> Result<Array2<f64>, String>
Finish the pass, returning the k×k border Gram. Errors if any rank’s
sequence is incomplete. The result is a pure function of the row content
and (n_rows, chunk_size) — identical bits for any node count, any
arrival interleaving, and any checkpoint/resume history on either side.
Auto Trait Implementations§
impl Freeze for CrossNodeGramReduction
impl RefUnwindSafe for CrossNodeGramReduction
impl Send for CrossNodeGramReduction
impl Sync for CrossNodeGramReduction
impl Unpin for CrossNodeGramReduction
impl UnsafeUnpin for CrossNodeGramReduction
impl UnwindSafe for CrossNodeGramReduction
Blanket Implementations§
impl<T> Allocation for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> DistributionExt for Twhere
T: ?Sized,
impl<T> DistributionExt for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> Pointable for T
impl<T> Pointable for T
impl<T> Read<Exclusive, BecauseExclusive> for Twhere
T: ?Sized,
Source§impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
Source§fn to_subset(&self) -> Option<SS>
fn to_subset(&self) -> Option<SS>
self from the equivalent element of its
superset. Read moreSource§fn is_in_subset(&self) -> bool
fn is_in_subset(&self) -> bool
self is actually part of its subset T (and can be converted to it).Source§fn to_subset_unchecked(&self) -> SS
fn to_subset_unchecked(&self) -> SS
self.to_subset but without any property checks. Always succeeds.Source§fn from_subset(element: &SS) -> SP
fn from_subset(element: &SS) -> SP
self to the equivalent element of its superset.