Struct KFACUtils 

pub struct KFACUtils;

K-FAC utilities for layer-specific operations

Implementations

impl KFACUtils

pub fn conv_kfac_update<T: Float + ScalarOperand + 'static>( input_patches: &Array2<T>, output_gradients: &Array2<T>, kernel_size: (usize, usize), _stride: (usize, usize), _padding: (usize, usize), ) -> Result<Array2<T>>

Compute the K-FAC weight-gradient statistic for a convolutional layer from already-extracted patches — the same quantity super::layer_state::KFACLayerState::weight_gradient computes for dense layers. This is the un-preconditioned gradient; feed the result through super::core::KFAC::apply_update_weight (G⁻¹ · grad · A⁻¹) to get the actual K-FAC parameter update.

input_patches is the im2col-style patch matrix [samples, in_channels * kh * kw] (one row per (batch element, output spatial location) pair), optionally with one extra trailing bias column, and output_gradients is the matching per-location output gradient [samples, out_channels]; the two must therefore share the same row count.

kernel_size, stride and padding describe how the caller produced input_patches and are accepted as provenance metadata / a light consistency check on input_patches’s column count. This function has no super::config::LayerInfo to consult, so — unlike the dense path — it cannot know whether the caller included a bias column; kernel_size is therefore checked loosely (with or without one trailing bias column) rather than rejected outright. stride/padding do not otherwise enter the computation: the covariance of already-extracted patches doesn’t depend on how they were extracted. This function does not perform patch extraction itself — pass already-extracted patches.

The result is output_gradients^T @ input_patches / samples, the same [out_features, in_features] convention used by super::layer_state::KFACLayerState::weight_gradient for dense layers.

Errors

Returns crate::error::OptimError::DimensionMismatch when input_patches and output_gradients disagree on the sample count, or crate::error::OptimError::InvalidParameter when input_patches’s column count is not kernel_size.0 * kernel_size.1 * in_channels for any positive in_channels, with or without one trailing bias column.

pub fn batchnorm_statistics<T: Float + FromPrimitive>( input: &Array2<T>, eps: T, ) -> Result<(Array1<T>, Array1<T>)>

Compute batch normalization statistics for K-FAC

pub fn grouped_conv_kfac<T: Float + ScalarOperand>( input: &Array2<T>, gradients: &Array2<T>, num_groups: usize, ) -> Result<Array2<T>>

Compute K-FAC update for grouped convolution layers

pub fn eigenvalue_regularization<T: Float + Debug + Send + Sync + 'static>( matrix: &Array2<T>, min_eigenvalue: T, ) -> Array2<T>

Compute eigenvalue-based regularization

pub fn kronecker_product_approx<T: Float + Debug + Send + Sync + 'static>( a: &Array2<T>, b: &Array2<T>, ) -> Array2<T>

Compute Kronecker product approximation for two matrices

pub fn trace<T: Float + Debug + Send + Sync + 'static>(matrix: &Array2<T>) -> T

Compute trace of a matrix

pub fn frobenius_norm<T: Float + Sum>(matrix: &Array2<T>) -> T

Compute Frobenius norm of a matrix

pub fn matrices_approx_equal<T: Float + Debug + Send + Sync + 'static>( a: &Array2<T>, b: &Array2<T>, tolerance: T, ) -> bool

Check if two matrices are approximately equal

pub fn exponential_moving_average<T: Float + Debug + Send + Sync + 'static>( current_value: T, new_value: T, decay: T, ) -> T

Compute running average with exponential decay

pub fn clamp_eigenvalues<T: Float + Debug + Send + Sync + 'static>( eigenvalues: &mut Array1<T>, min_val: T, max_val: T, )

Clamp eigenvalues to prevent numerical instability

pub fn condition_number_svd_approx<T: Float + Debug + Send + Sync + 'static>( matrix: &Array2<T>, ) -> T

Compute condition number using singular values (approximation)

pub fn diag_matrix<T: Float + Clone>(diagonal: &Array1<T>) -> Array2<T>

Extract diagonal elements and create diagonal matrix

pub fn symmetrize<T: Float + Debug + Send + Sync + 'static>( matrix: &Array2<T>, ) -> Array2<T>

Symmetrize a matrix: (A + A^T) / 2