Struct TransformerOptimizer 

pub struct TransformerOptimizer<T: Float + Debug + ScalarOperand + Send + Sync + 'static> { /* private fields */ }

Transformer-based meta-learning optimizer

Implementations

impl<T: Float + Debug + Send + Sync + 'static + ScalarOperand + FromPrimitive> TransformerOptimizer<T>

pub fn new(config: TransformerBasedOptimizerConfig<T>) -> Result<Self>

Create new transformer optimizer.

Errors

Returns OptimError::InvalidConfig if config (or any of its three sub-configurations — memory, meta-learning, performance) is inconsistent.

F46: TransformerBasedOptimizerConfig::validate and the three sub-config validators it delegates to existed and were correct, but nothing in the crate ever called them. A model_dimension of 0, an attention_head_dimension · num_attention_heads that did not equal model_dimension, a dropout_rate of 2.0 or a zero batch_size were all accepted here and then produced either a wrong answer or a panic several layers down. The check now runs before anything is allocated.

pub fn generate_optimization_step( &mut self, gradient_history: &Array2<T>, parameter_history: &Array2<T>, loss_history: &Array1<T>, ) -> Result<Array1<T>>

Generate optimization step using transformer

pub fn train_on_trajectories( &mut self, trajectories: &[OptimizationTrajectory<T>], ) -> Result<TrainingMetrics>

Train the transformer on optimization trajectories

pub fn forward_sequence(&mut self, sequence: &Array2<T>) -> Result<Array2<T>>

Forward pass through the transformer for a sequence (inference path).

pub fn apply_architecture_config( &mut self, config: &TransformerBasedOptimizerConfig<T>, ) -> Result<ArchitectureUpdate>

Adopt a new architecture configuration.

The return value says exactly what happened, because the two cases differ in a way callers must not be left guessing about:

• ArchitectureUpdate::Unchanged — the proposal matched the live configuration; nothing was touched.
• ArchitectureUpdate::InPlace — only non-structural fields differed (learning_rate, dropout_rate, clipping, weight decay). They were adopted and all learned weights were kept.
• ArchitectureUpdate::Rebuilt — a structural dimension differed (model_dimension, num_transformer_layers, num_attention_heads, attention_head_dimension, feedforward_dimension, sequence_length, activation_function, positional_encoding_type). Weights of one shape cannot be reinterpreted at another, so the transformer stack, positional encoding, meta-learning networks and optimizer state were rebuilt from a fresh initialization — every learned parameter was discarded.

That last case is destructive, which is why it is reported rather than signalled by a bare true. AdaptiveTransformerEnhancement’s architecture proposal is a function of the measured landscape rather than of the current architecture, so repeated enhancement converges and stops rebuilding instead of wiping the weights on every call.

This is also what makes AdaptiveTransformerEnhancement::enhance_optimizer actually enhance the optimizer it is given; before, its &mut TransformerOptimizer argument was never touched.

Errors

Propagates construction errors from the new configuration (e.g. a head count that does not divide the model dimension), leaving self unchanged.

pub fn config(&self) -> &TransformerBasedOptimizerConfig<T>

The configuration currently in force.

pub fn parameter_count(&self) -> usize

Total number of learnable parameters this optimizer owns.

Exactly the transformer architecture plus the meta-learning adaptation networks — the only two subsystems that hold weights.

pub fn get_state(&self) -> &TransformerOptimizerState<T>

Get current state

pub fn get_performance_metrics(&self) -> &TransformerPerformanceTracker<T>

Get performance metrics

pub fn reset_state(&mut self) -> Result<()>

Reset optimizer state