pub struct GradientAccumulator {
pub dimension: usize,
pub clip_strategy: ClipStrategy,
pub pending: Vec<PeerGradient>,
pub stats: AccumulatorStats,
}Expand description
Accumulates gradients from multiple federated peers and produces a weighted-averaged, optionally-clipped aggregate on each commit.
Fields§
§dimension: usizeExpected length of every gradient vector.
clip_strategy: ClipStrategyClipping strategy applied after averaging.
pending: Vec<PeerGradient>Gradients waiting to be committed.
stats: AccumulatorStatsRunning statistics updated on each successful commit.
Implementations§
Source§impl GradientAccumulator
impl GradientAccumulator
Sourcepub fn new(dimension: usize, clip_strategy: ClipStrategy) -> Self
pub fn new(dimension: usize, clip_strategy: ClipStrategy) -> Self
Create a new accumulator expecting gradients of length dimension.
Sourcepub fn add(&mut self, pg: PeerGradient) -> Result<(), AccumulationError>
pub fn add(&mut self, pg: PeerGradient) -> Result<(), AccumulationError>
Add a peer gradient to the pending queue.
Returns AccumulationError::DimensionMismatch when
pg.gradient.len() != self.dimension.
Sourcepub fn commit(&mut self) -> Result<Vec<f32>, AccumulationError>
pub fn commit(&mut self) -> Result<Vec<f32>, AccumulationError>
Commit all pending gradients, producing a weighted average.
After a successful commit the pending queue is cleared and
AccumulatorStats is updated.
§Errors
AccumulationError::EmptyAccumulator— no pending gradients.AccumulationError::WeightSumZero— all weights are0.0.
Sourcepub fn apply_clip(&self, grad: &mut [f32]) -> bool
pub fn apply_clip(&self, grad: &mut [f32]) -> bool
Apply the accumulator’s ClipStrategy to grad in-place.
Returns true if any modification was made.
Sourcepub fn pending_count(&self) -> usize
pub fn pending_count(&self) -> usize
Number of gradients currently waiting to be committed.
Sourcepub fn stats(&self) -> &AccumulatorStats
pub fn stats(&self) -> &AccumulatorStats
Borrow the current statistics.
Trait Implementations§
Auto Trait Implementations§
impl Freeze for GradientAccumulator
impl RefUnwindSafe for GradientAccumulator
impl Send for GradientAccumulator
impl Sync for GradientAccumulator
impl Unpin for GradientAccumulator
impl UnsafeUnpin for GradientAccumulator
impl UnwindSafe for GradientAccumulator
Blanket Implementations§
impl<T> Allocation for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more