pub struct OuterAvg;Expand description
Identity outer optimizer: new_global = consensus. Stateless, no
momentum, no checkpoint artifact. This is the default and reproduces the
pre-outer-optimizer averaging behavior exactly.
Trait Implementations§
impl Copy for OuterAvg
Source§impl OuterOptimizer for OuterAvg
impl OuterOptimizer for OuterAvg
Source§fn outer_step(
&mut self,
_prev_global: &[Tensor],
consensus: &[Tensor],
) -> Result<Vec<Tensor>>
fn outer_step( &mut self, _prev_global: &[Tensor], consensus: &[Tensor], ) -> Result<Vec<Tensor>>
Transform
consensus (this window’s work-weighted average) into the
new global parameters, given prev_global (the global adopted at the
end of the previous window). Momentum is updated internally. Read moreSource§fn checkpoint_state(&self) -> Option<Vec<Tensor>>
fn checkpoint_state(&self) -> Option<Vec<Tensor>>
Snapshot the checkpointable outer state (the slow momentum), one
tensor per model parameter in
Module::parameters() order, for
<stem>.outer.fdl. None for stateless variants (OuterAvg),
which write no artifact. Returns clones (shallow); the caller
serializes them synchronously, so the live state may be overwritten by
the next window without racing the detached write.Source§fn load_checkpoint_state(&mut self, _state: Vec<Tensor>) -> Result<()>
fn load_checkpoint_state(&mut self, _state: Vec<Tensor>) -> Result<()>
Restore outer state from
<stem>.outer.fdl on resume. state is
positional, one tensor per parameter (Module::parameters() order).
Stateless variants ignore it.Source§fn resets_inner(&self) -> bool
fn resets_inner(&self) -> bool
Whether this variant requires disposable inner optimizer state:
the worker, each sync, fully overwrites its params with the new global
and resets its inner optimizer (clears momentum, step count). This
is DiLoCo’s contract — the inner optimizer is restarted every outer
round, which is what makes the outer momentum the canonical,
resume-faithful state.
false (default) keeps the inner loop
continuous (OuterAvg, SlowMomentum). Read moreAuto Trait Implementations§
impl Freeze for OuterAvg
impl RefUnwindSafe for OuterAvg
impl Send for OuterAvg
impl Sync for OuterAvg
impl Unpin for OuterAvg
impl UnsafeUnpin for OuterAvg
impl UnwindSafe for OuterAvg
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more