Module ml 

Machine learning loss functions and optimizer state types. ML Toolkit — loss functions, optimizers, activations, metrics.

Determinism Contract

• All functions are deterministic (no randomness except seeded kfold).
• Kahan summation for all reductions.
• Stable sort for AUC-ROC with index tie-breaking.

Structs

AdamState

Adam optimizer state.

ConfusionMatrix

Binary confusion matrix.

EarlyStoppingState

Early stopping state tracker.

LbfgsState

L-BFGS optimizer state.

SgdState

SGD optimizer state.

Functions

accuracy

Accuracy: (TP + TN) / total.

adam_step

Adam step: sequential, deterministic.

apply_dropout

Apply dropout: element-wise multiply data by mask.

auc_roc

AUC-ROC via trapezoidal rule. DETERMINISM: sort by score with stable sort + index tie-breaking.

batch_indices

Creates deterministic batch index ranges for mini-batch training.

batch_norm

Batch normalization (inference mode). y = gamma * (x - running_mean) / sqrt(running_var + eps) + beta.

binary_cross_entropy

Binary cross-entropy: -sum(t*ln(p) + (1-t)*ln(1-p)) / n.

bootstrap

Bootstrap confidence interval for a statistic (e.g., mean). Returns (point_estimate, ci_lower, ci_upper, standard_error). stat_fn is 0=mean, 1=median.

confusion_matrix

Build confusion matrix from predicted and actual boolean labels.

cross_entropy_loss

Cross-entropy loss: -sum(target * ln(pred + eps)) / n.

dropout_mask

Dropout mask generation using seeded RNG for determinism. Returns mask of 0.0 and scale values (1/(1-p)) using inverted dropout.

embedding

Embedding lookup: maps integer indices to dense vectors.

f1_score

F1 score: 2 * (precision * recall) / (precision + recall).

gru_cell

GRU cell forward pass.

gru_cell_fused

Fused GRU cell: minimizes intermediate tensor allocations.

hinge_loss

Hinge loss: sum(max(0, 1 - target * pred)) / n.

huber_loss

Huber loss: quadratic for small errors, linear for large.

kfold_indices

K-fold cross-validation indices. DETERMINISM: uses seeded RNG (Fisher-Yates).

l1_grad

L1 regularization gradient: lambda * sign(params).

l1_penalty

L1 regularization penalty: lambda * sum(|params|).

l2_grad

L2 regularization gradient: lambda * params.

l2_penalty

L2 regularization penalty: 0.5 * lambda * sum(params^2).

lbfgs_step

L-BFGS step with strong Wolfe line search.

lr_cosine

Learning rate schedule: cosine annealing. lr = min_lr + 0.5 * (max_lr - min_lr) * (1 + cos(pi * epoch / total_epochs))

lr_linear_warmup

Learning rate schedule: linear warmup. lr = initial_lr * min(1.0, epoch / warmup_epochs).

lr_step_decay

Learning rate schedule: step decay. lr = initial_lr * decay_rate^(floor(epoch / step_size))

lstm_cell

LSTM cell forward pass.

lstm_cell_fused

Fused LSTM cell: minimizes intermediate tensor allocations.

mse_loss

Mean Squared Error: sum((pred - target)^2) / n.

multi_head_attention

Multi-head attention: Q, K, V projections + scaled dot-product attention + output projection.

pca

Principal Component Analysis via SVD of centered data.

permutation_test

Permutation test: test whether two groups differ on a statistic. Returns (observed_diff, p_value).

precision

Precision: TP / (TP + FP).

recall

Recall / sensitivity: TP / (TP + FN).

sgd_step

SGD step: sequential, deterministic.

stratified_split

Stratified train/test split: maintains class proportions in both sets. labels is an array of integer class labels, test_frac is fraction for test set. Returns (train_indices, test_indices).

train_test_split

Train/test split indices.

wolfe_line_search

Strong Wolfe line search.