Struct TransformerOptimizer 

pub struct TransformerOptimizer<T: Float + Debug + Default + Clone + Sum + ScalarOperand + Send + Sync + 'static> { /* private fields */ }

Transformer-based neural optimizer with self-attention mechanisms

Implementations

impl<T: Float + Debug + Default + Clone + Sum + ScalarOperand + Send + Sync + 'static> TransformerOptimizer<T>

pub fn new(config: TransformerOptimizerConfig) -> Result<Self>

Create new transformer optimizer

pub fn step( &mut self, parameters: &mut HashMap<String, Array2<T>>, gradients: &mut HashMap<String, Array2<T>>, loss: T, ) -> Result<T>

Perform one optimization step, writing the updated values back into parameters in place.

The pipeline is: regularize -> process -> adapt learning rate -> integrate momentum -> run the recent optimization history through the transformer -> apply param -= lr * modulation * momentum_gradient.

The transformer output is used as a bounded, strictly positive per-coordinate scale on the momentum-integrated gradient rather than as the update itself. This is the standard learned-preconditioner formulation: the network can speed up or slow down individual coordinates but cannot, by itself, reverse the descent direction, so an untrained network degrades gracefully to plain gradient descent instead of diverging.

Returns the loss it was given, so callers can chain steps.

pub fn get_statistics(&self) -> HashMap<String, f64>

Get current optimization statistics

pub fn set_training(&mut self, training: bool)

Switch the network between training mode (dropout active) and inference mode. Optimizers are created in inference mode.

pub fn set_learning_rate(&mut self, lr: T)

Override the base learning rate used by the adapter.

pub fn current_learning_rate(&self) -> T

Current (post-adaptation) learning rate.

pub fn step_count(&self) -> usize

Number of steps taken so far.

pub fn config(&self) -> &TransformerOptimizerConfig

Configuration in use.

pub fn lr_adapter_mut(&mut self) -> &mut LearningRateAdapter<T>

Mutable access to the learning rate adapter.

pub fn gradient_processor_mut(&mut self) -> &mut GradientProcessor<T>

Mutable access to the gradient processor.

pub fn momentum_integrator_mut(&mut self) -> &mut MomentumIntegrator<T>

Mutable access to the momentum integrator.

pub fn regularizer_mut(&mut self) -> &mut TransformerRegularizer<T>

Mutable access to the regularizer.

pub fn meta_learner_mut(&mut self) -> &mut TransformerMetaLearner<T>

Mutable access to the meta-learner.

pub fn evaluator_mut(&mut self) -> &mut TransformerEvaluator<T>

Mutable access to the evaluator.

pub fn reset(&mut self) -> Result<()>

Reset optimizer state

Trait Implementations

impl<T: Debug + Float + Debug + Default + Clone + Sum + ScalarOperand + Send + Sync + 'static> Debug for TransformerOptimizer<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.