Struct ParallelOptimizer 

pub struct ParallelOptimizer<O, A, D>
where
    O: Optimizer<A, D> + Clone + Send + Sync,
    A: Float + ScalarOperand + Debug + Send + Sync,
    D: Dimension,
{ /* private fields */ }

Parallel optimizer wrapper for processing multiple parameter groups

This wrapper enables parallel processing of multiple parameter groups, providing significant speedup on multi-core systems.

Examples

use scirs2_core::ndarray::Array1;
use optirs_core::optimizers::{SGD, Optimizer};
use optirs_core::parallel_optimizer::ParallelOptimizer;

// Create base optimizer
let optimizer = SGD::new(0.01);

// Wrap in parallel optimizer
let mut parallel_opt = ParallelOptimizer::new(optimizer);

// Process multiple parameter groups in parallel
let params_list = vec![
    Array1::zeros(1000),
    Array1::zeros(2000),
    Array1::zeros(1500),
];
let grads_list = vec![
    Array1::from_elem(1000, 0.1),
    Array1::from_elem(2000, 0.1),
    Array1::from_elem(1500, 0.1),
];

let updated = parallel_opt.step_parallel_groups(&params_list, &grads_list).expect("parallel_opt.step_parallel_groups succeeds");

State handling

Stateful optimizers (Adam, AdamW, LAMB, RAdam, Lion, …) keep per-parameter moment estimates. A dedicated optimizer instance is therefore materialized and retained for every parameter group, and each group’s instance is mutated in place across calls. Without this, every call would optimize with a freshly reset clone and, for example, Adam would degenerate into sign-SGD.

Implementations

impl<O, A, D> ParallelOptimizer<O, A, D>

where O: Optimizer<A, D> + Clone + Send + Sync, A: Float + ScalarOperand + Debug + Send + Sync, D: Dimension,

pub fn new(base_optimizer: O) -> Self

Creates a new parallel optimizer wrapper

Arguments

• base_optimizer - The base optimizer to parallelize

pub fn num_groups(&self) -> usize

Number of parameter groups for which persistent state is currently held

pub fn group_optimizer(&self, group: usize) -> Option<&O>

Access the persistent optimizer instance of a parameter group

pub fn group_optimizer_mut(&mut self, group: usize) -> Option<&mut O>

Access the persistent optimizer instance of a parameter group mutably

pub fn reset_group_state(&mut self)

Drop all per-group state, restarting every group from the base optimizer

pub fn step_parallel_groups( &mut self, params_list: &[Array<A, D>], grads_list: &[Array<A, D>], ) -> Result<Vec<Array<A, D>>>

where Array<A, D>: Clone + Send + Sync,

Process multiple parameter groups in parallel

This method distributes parameter groups across available CPU cores for parallel processing.

Arguments

• params_list - List of parameter arrays
• grads_list - List of gradient arrays

Returns

Updated parameter arrays processed in parallel

pub fn inner(&self) -> &O

Get the underlying optimizer

pub fn inner_mut(&mut self) -> &mut O

Get mutable reference to underlying optimizer

pub fn get_learning_rate(&self) -> A

Get the current learning rate from the base optimizer

pub fn set_learning_rate(&mut self, learning_rate: A)

Set the learning rate on the base optimizer and on every per-group instance

Trait Implementations

impl<O, A, D> Debug for ParallelOptimizer<O, A, D>

where O: Optimizer<A, D> + Clone + Send + Sync + Debug, A: Float + ScalarOperand + Debug + Send + Sync + Debug, D: Dimension + Debug,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.