Struct AdamW 

pub struct AdamW { /* private fields */ }

AdamW, as MLX’s optimizers.AdamW: Adam with decoupled weight decay (MLX’s default 0.01).

Implementations

impl AdamW

pub fn new(learning_rate: impl Into<LearningRate>) -> Self

AdamW with MLX’s defaults: betas (0.9, 0.999), eps = 1e-8, weight decay 0.01.

pub fn betas(self, b1: f64, b2: f64) -> Self

Sets the betas.

pub fn eps(self, eps: f64) -> Self

Sets eps, added to the denominator.

pub fn weight_decay(self, weight_decay: f64) -> Self

Sets the decoupled weight decay.

pub fn bias_correction(self) -> Self

Enables bias correction.

Trait Implementations

impl Clone for AdamW

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AdamW

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Leaves for AdamW

fn visit(&self, __v: &mut Visit<'_>)

Reports every leaf (and child tree, enum variant, mode flag and static field) to v, in order.

impl Optimizer for AdamW

fn apply( &mut self, params: &[(&str, &Array)], grads: &[&Array], ) -> Result<Vec<Array>>

One step over named parameters (the path keys each parameter’s state) and their gradients: the updated parameters, in order. Increments the step once.

fn init_params(&mut self, params: &[(&str, &Array)])

Creates zero state for params that have none (paths with state are kept).

fn visit_state(&self, v: &mut Visit<'_>)

Visits the state (step, learning_rate, slots) and the hyperparameters.

fn visit_state_mut(&mut self, v: &mut VisitMut<'_>)

Visits the state mutably.

fn learning_rate(&self) -> Array

The learning rate: the 0-D float32 array in the state.

fn set_learning_rate(&mut self, lr: f32)

Sets the learning rate array (in a compiled step it is an input: no retrace).

fn clone_box(&self) -> Box<dyn Optimizer>

A boxed copy (object-safe Clone).

fn init<M: Tree>(&mut self, model: &M)

where Self: Sized,

Creates the state of every trainable leaf of model now (instead of at the first update), so a compiled step’s state structure is fixed from the start; also needed before loading a checkpoint into a fresh optimizer.

fn update<M: Tree>(&mut self, model: &mut M, grads: &M) -> Result<()>

where Self: Sized,

Applies one step to the trainable leaves of model from grads, a gradient tree of the model’s type with the model’s paths (else an error). Frozen parameters and buffers are left alone; their state is kept.

impl Tree for AdamW

where Self: Clone,

fn visit_mut(&mut self, __v: &mut VisitMut<'_>)

Reports every leaf (and child tree and mode flag) to v mutably, in the order of Leaves::visit.

fn for_each(&self, f: impl FnMut(&str, &Array))

Calls f(path, leaf) for every leaf, in order.

fn map(&self, f: impl FnMut(&str, &Array) -> Array) -> Self

The tree with every leaf replaced by f(path, leaf) (same structure; parameters keep their frozen flag).

fn leaves(&self) -> Vec<Array>

The leaves, in order.

fn paths(&self) -> Vec<String>

The leaves’ paths, in order.

fn flatten(&self) -> Vec<(String, Array)>

The leaves with their paths, in order (MLX’s tree_flatten).

fn unflatten( &self, leaves: impl IntoIterator<Item = Array>, ) -> Result<Self, Error>

A tree of this structure (self is the template) with leaves in order: exactly Tree::len of them, else an error naming both counts.

fn load_flat( &mut self, named: impl IntoIterator<Item = (String, Array)>, mode: LoadMode, ) -> Result<(), Error>

Replaces leaves by path. LoadMode::Strict requires exactly the tree’s paths with matching shapes; LoadMode::Partial assigns the matching names and ignores unknown ones (shapes must still match). Loaded dtypes are kept. On any problem the error lists every one (missing, unknown and mismatched names, sorted) and the tree is unchanged. Duplicate paths (in the tree or in named) are an error.

fn len(&self) -> usize

The number of leaves.

fn is_empty(&self) -> bool

Whether the tree has no leaves.

fn def(&self) -> TreeDef

The tree’s structure.