Struct Fp32MasterOptimizer 

pub struct Fp32MasterOptimizer<O> { /* private fields */ }

Opt-in FP32 master parameters for an existing simple optimizer.

The wrapped optimizer receives FP32 parameters and gradients; its algorithm, hyperparameters and per-parameter state are unchanged. Returned parameters retain their incoming storage dtype. Existing optimizers do not opt in.

Implementations

impl<O> Fp32MasterOptimizer<O>

pub fn new(optimizer: O) -> Self

Wrap an existing optimizer without modifying its configuration.

pub fn with_grad_clipping(self, clipping: GradientClipping) -> Self

Explicitly apply RUDA’s existing per-parameter clipping after FP32 conversion.

pub fn with_gradient_scale(self, scale: f32) -> Self

Explicitly divide gradients by this loss scale in FP32 before clipping.

The caller scales the loss and uses one scale for the accumulation window. No dynamic scaling, non-finite policy or optimizer-step skipping is enabled.

pub fn init<B, M>(self) -> OptimizerAdaptor<Self, M, B>

where B: AutodiffBackend, M: AutodiffModule<B>, O: SimpleOptimizer<B::InnerBackend>,

Build RUDA’s original module adaptor, preserving parameter IDs and records. Clipping is disabled unless explicitly configured on the wrapper or adaptor.

Trait Implementations

impl<O: Clone> Clone for Fp32MasterOptimizer<O>

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<B: Backend, O: SimpleOptimizer<B>> SimpleOptimizer<B> for Fp32MasterOptimizer<O>

type State<const D: usize> = Fp32MasterState<B, D, <O as SimpleOptimizer<B>>::State<D>>

The state of the optimizer. It also implements record, so that it can be saved.

fn step<const D: usize>( &self, lr: LearningRate, tensor: Tensor<B, D>, grad: Tensor<B, D>, state: Option<Self::State<D>>, ) -> (Tensor<B, D>, Option<Self::State<D>>)

The optimizer step is performed for one tensor at a time with its gradient and state.

fn to_device<const D: usize>( state: Self::State<D>, device: &B::Device, ) -> Self::State<D>

Change the device of the state.