Struct KronPsgd 

pub struct KronPsgd {
    pub lr: f32,
    pub precond_lr: f32,
    pub momentum: f32,
    pub weight_decay: f32,
    pub eps: f32,
    pub clip: f32,
    /* private fields */
}

Kron-PSGD — Kronecker-factored preconditioned SGD.

Fields

lr: f32

Learning rate.

precond_lr: f32

Learning rate for the preconditioner update (Lie-group descent on Q_L / Q_R). Default 0.1. Too high ⇒ Q drifts; too low ⇒ preconditioner lags.

momentum: f32

Polyak momentum for the preconditioned-gradient SGD step. Default 0.9.

weight_decay: f32

L2 weight-decay coefficient (folded into the gradient). Default 0.0.

eps: f32

Numerical floor on the preconditioner-update normalizer. Default 1e-8.

clip: f32

Cap the per-coordinate magnitude of the preconditioned update (defensive — early Q estimates can be ill-conditioned). Default 1.0.

Implementations

impl KronPsgd

pub fn new(lr: f32) -> Self

Construct with (precond_lr, μ, λ, ε, clip) = (0.1, 0.9, 0.0, 1e-8, 1.0).

pub fn with_momentum(self, mu: f32) -> Self

Override the Polyak momentum.

pub fn with_weight_decay(self, wd: f32) -> Self

Override the weight-decay coefficient.

Trait Implementations

impl Clone for KronPsgd

fn clone(&self) -> KronPsgd

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for KronPsgd

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for KronPsgd

fn step(&mut self, name: &str, shape: &[usize], param: &mut [f32], grad: &[f32])

fn end_iteration(&mut self)

Advance the global step counter. Most algorithms increment per call to [step], so most implementations leave this a no-op.

fn lr_scale(&self, _name: &str) -> f32

Per-tensor multiplier on the effective learning rate. Default is 1.0 for every name. Override when wrapping this crate to support per-name LR schedules (e.g. embedding-vs-attention splits, or the Gaussian-splat attribute-typed LR setup). The CPU impls in this crate currently honor this only when the caller passes a pre-scaled lr for the relevant call — backends are encouraged to consult it inside their fused kernel.