Struct MultiOptimizer 

pub struct MultiOptimizer { /* private fields */ }

Parameter groups, as MLX’s MultiOptimizer: each group’s filter selects parameters by path (the first matching group wins) and its optimizer updates them; the default optimizer takes the rest. Every group steps once per update. Its state is a tree: groups.0.…, …, default.….

use nuvec_nn::optim::{Adam, MultiOptimizer, Sgd};

let opt = MultiOptimizer::new(Sgd::new(1e-2).weight_decay(1e-4))
    .group(|path| path.contains("lora_a"), Adam::new(1e-3));

Implementations

impl MultiOptimizer

pub fn new(default: impl Optimizer + 'static) -> Self

Groups with default for every parameter no group selects.

pub fn group( self, filter: impl Fn(&str) -> bool + Send + Sync + 'static, opt: impl Optimizer + 'static, ) -> Self

Adds a group: opt updates the parameters whose path matches filter (and no earlier group’s).

Trait Implementations

impl Clone for MultiOptimizer

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MultiOptimizer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Leaves for MultiOptimizer

fn visit(&self, __v: &mut Visit<'_>)

Reports every leaf (and child tree, enum variant, mode flag and static field) to v, in order.

impl Optimizer for MultiOptimizer

fn apply( &mut self, params: &[(&str, &Array)], grads: &[&Array], ) -> Result<Vec<Array>>

One step over named parameters (the path keys each parameter’s state) and their gradients: the updated parameters, in order. Increments the step once.

fn init_params(&mut self, params: &[(&str, &Array)])

Creates zero state for params that have none (paths with state are kept).

fn visit_state(&self, v: &mut Visit<'_>)

Visits the state (step, learning_rate, slots) and the hyperparameters.

fn visit_state_mut(&mut self, v: &mut VisitMut<'_>)

Visits the state mutably.

fn learning_rate(&self) -> Array

The learning rate: the 0-D float32 array in the state.

fn set_learning_rate(&mut self, lr: f32)

Sets the learning rate array (in a compiled step it is an input: no retrace).

fn clone_box(&self) -> Box<dyn Optimizer>

A boxed copy (object-safe Clone).

fn init<M: Tree>(&mut self, model: &M)

where Self: Sized,

Creates the state of every trainable leaf of model now (instead of at the first update), so a compiled step’s state structure is fixed from the start; also needed before loading a checkpoint into a fresh optimizer.

fn update<M: Tree>(&mut self, model: &mut M, grads: &M) -> Result<()>

where Self: Sized,

Applies one step to the trainable leaves of model from grads, a gradient tree of the model’s type with the model’s paths (else an error). Frozen parameters and buffers are left alone; their state is kept.

impl Tree for MultiOptimizer

where Self: Clone,

fn visit_mut(&mut self, __v: &mut VisitMut<'_>)

Reports every leaf (and child tree and mode flag) to v mutably, in the order of Leaves::visit.

fn for_each(&self, f: impl FnMut(&str, &Array))

Calls f(path, leaf) for every leaf, in order.

fn map(&self, f: impl FnMut(&str, &Array) -> Array) -> Self

The tree with every leaf replaced by f(path, leaf) (same structure; parameters keep their frozen flag).

fn leaves(&self) -> Vec<Array>

The leaves, in order.

fn paths(&self) -> Vec<String>

The leaves’ paths, in order.

fn flatten(&self) -> Vec<(String, Array)>

The leaves with their paths, in order (MLX’s tree_flatten).

fn unflatten( &self, leaves: impl IntoIterator<Item = Array>, ) -> Result<Self, Error>

A tree of this structure (self is the template) with leaves in order: exactly Tree::len of them, else an error naming both counts.

fn load_flat( &mut self, named: impl IntoIterator<Item = (String, Array)>, mode: LoadMode, ) -> Result<(), Error>

Replaces leaves by path. LoadMode::Strict requires exactly the tree’s paths with matching shapes; LoadMode::Partial assigns the matching names and ignores unknown ones (shapes must still match). Loaded dtypes are kept. On any problem the error lists every one (missing, unknown and mismatched names, sorted) and the tree is unchanged. Duplicate paths (in the tree or in named) are an error.

fn len(&self) -> usize

The number of leaves.

fn is_empty(&self) -> bool

Whether the tree has no leaves.

fn def(&self) -> TreeDef

The tree’s structure.