Crate rkhs 

rkhs

Kernel methods.

Why “RKHS”?

A Reproducing Kernel Hilbert Space is the mathematical structure where kernel methods live. Every positive-definite kernel k(x,y) defines an RKHS (via Mercer’s theorem), and every RKHS has a unique reproducing kernel.

This crate provides kernels, Gram matrices, MMD, and kernel quantile embeddings. A small set of dense associative-memory functions is re-exported from hopfield for compatibility with earlier examples.

Intuition

Kernels measure similarity in a (potentially infinite-dimensional) feature space without ever computing the features explicitly. This “kernel trick” enables nonlinear methods with linear complexity.

MMD (Maximum Mean Discrepancy) uses kernels to test whether two samples come from the same distribution. It embeds distributions into an RKHS and measures the distance between their mean embeddings (kernel mean embeddings).

Key Functions

Function	Purpose
rbf	Radial Basis Function (Gaussian) kernel
epanechnikov	Optimal kernel for density estimation
matern_32	Matérn kernel (ν = 3/2; also matern_12, matern_52)
polynomial	Polynomial kernel
kernel_matrix	Gram matrix K[i,j] = k(x_i, x_j)
mmd_biased	O(n²) biased MMD estimate
mmd_unbiased	O(n²) unbiased MMD u-statistic
mmd_permutation_test	Significance test via permutation
kernel_quantile_embedding	Kernel embedding at a quantile level
qmmd	Quantile MMD (tail-sensitive distribution comparison)
weighted_qmmd	QMMD with configurable quantile-level weighting
quantile_function_embedding	Kernel-smoothed quantile function at specified levels
quantile_distribution_kernel	Kernel between distributions via quantile embeddings
quantile_gram_matrix	Gram matrix restricted to a quantile level

Quick Start (MMD)

use rkhs::{rbf, mmd_unbiased};

let x = vec![vec![0.0, 0.0], vec![0.1, 0.1], vec![0.2, 0.0]];
let y = vec![vec![5.0, 5.0], vec![5.1, 5.1], vec![5.2, 5.0]];

// Different distributions → large MMD
let mmd = mmd_unbiased(&x, &y, |a, b| rbf(a, b, 1.0));
assert!(mmd > 0.5);

Why Kernels Matter for ML

• GAN evaluation: FID uses MMD-like statistics to compare generated vs real
• Domain adaptation: Minimize MMD between source and target distributions
• Two-sample testing: Detect distribution shift in production systems
• Kernel regression: Nonparametric regression via kernel mean embedding

What Can Go Wrong

1. Bandwidth too small: RBF kernel becomes nearly diagonal, loses structure.
2. Bandwidth too large: Everything becomes similar, no discrimination.
3. Numerical instability: Very large distances → exp(-large) → 0 underflow.
4. MMD variance: With small samples, MMD estimates are noisy. Use permutation test.
5. Kernel not characteristic: Not all kernels can distinguish all distributions. RBF is characteristic (good); polynomial is not (bad for two-sample test).

References

• Gretton et al. (2012). “A Kernel Two-Sample Test” (JMLR)
• Muandet et al. (2017). “Kernel Mean Embedding of Distributions” (Found. & Trends)
• Naslidnyk et al. (2025). “Kernel Quantile Embeddings”

Re-exports

pub use clam::am_assign;Deprecated

pub use clam::am_contract;Deprecated

pub use clam::am_soft_assign;Deprecated

pub use clam::clam_loss;Deprecated

pub use distribution_kernel::expected_likelihood_kernel;

pub use distribution_kernel::fisher_kernel_categorical;

pub use distribution_kernel::jensen_shannon_kernel;

pub use distribution_kernel::probability_product_kernel;

pub use graph_kernel::random_walk_kernel;Deprecated

pub use graph_kernel::sliced_wasserstein_graph_kernel;Deprecated

pub use graph_kernel::structural_node_features;Deprecated

pub use graph_kernel::wl_subtree_kernel;Deprecated

pub use krr_memory::KrrMemory;

pub use quantile_kernel::kernel_quantile_embedding;

pub use quantile_kernel::qmmd;

pub use quantile_kernel::quantile_distribution_kernel;

pub use quantile_kernel::quantile_function_embedding;

pub use quantile_kernel::quantile_gram_matrix;

pub use quantile_kernel::weighted_qmmd;

pub use quantile_kernel::QuantileWeight;

Modules

clamDeprecated

CLAM: Clustering with Associative Memory helpers.

distribution_kernel

Kernels on probability distributions. Distribution kernels: similarity measures between probability distributions.

graph_kernel

Kernels on labeled graphs.

krr_memory

Kernel Ridge Regression associative memory (high-capacity Hopfield network). Kernel Ridge Regression associative memory (Hopfield network), per “Kernel Ridge Regression for Efficient Learning of High-Capacity Hopfield Networks” (arXiv:2504.12561) and its KLR sibling (arXiv:2504.07633).

quantile_kernel

Kernel quantile embeddings for tail-sensitive distribution comparison. Kernel quantile embeddings for tail-sensitive distribution comparison.

Functions

cosine

Cosine kernel: k(x, y) = cos(π/2 * min(||x-y||/σ, 1))

energy_lse

Log-Sum-Exp (LSE) energy for Dense Associative Memory.

energy_lse_grad

Gradient of LSE energy: ∇_v E_LSE(v; Ξ)

energy_lsr

Log-Sum-ReLU (LSR) energy for Dense Associative Memory with Epanechnikov kernel.

energy_lsr_grad

Gradient of LSR energy: ∇_v E_LSR(v; Ξ)

epanechnikov

Epanechnikov kernel: k(x, y) = max(0, 1 - ||x-y||² / σ²)

kernel_matrix

Compute the Gram matrix K[i,j] = k(X[i], X[j]).

kernel_sum

Compute kernel sum: Σ_μ κ(v, ξ^μ)

laplacian

Laplacian kernel: k(x, y) = exp(-||x-y||₁ / σ)

linear

Linear kernel: k(x, y) = ⟨x, y⟩

matern_12

Matérn kernel with smoothness ν = 1/2 (the exponential kernel): k(x, y) = exp(-d / ℓ), where d = ||x - y||₂ and ℓ is the lengthscale.

matern_32

Matérn kernel with smoothness ν = 3/2: k(x, y) = (1 + √3·d/ℓ)·exp(-√3·d/ℓ), where d = ||x - y||₂.

matern_52

Matérn kernel with smoothness ν = 5/2: k(x, y) = (1 + √5·d/ℓ + 5d²/(3ℓ²))·exp(-√5·d/ℓ), where d = ||x - y||₂.

median_bandwidth

Median heuristic for RBF bandwidth selection.

mmd_biased

Biased MMD estimate in O(n^2) time.

mmd_permutation_test

Permutation test for MMD significance.

mmd_unbiased

Unbiased MMD² estimate (u-statistic).

polynomial

Polynomial kernel: k(x, y) = (γ⟨x,y⟩ + c)^d

quartic

Quartic (Biweight) kernel: k(x, y) = max(0, 1 - ||x-y||² / σ²)²

rbf

Radial Basis Function (Gaussian) kernel: k(x, y) = exp(-||x-y||² / (2σ²))

rbf_kernel_matrix_ndarray

RBF Gram matrix for an ndarray matrix of points.

retrieve_memory

Retrieve memory using energy descent.

triangle

Triangle kernel: k(x, y) = max(0, 1 - ||x-y|| / σ)

tricube

Tricube kernel: k(x, y) = max(0, 1 - (||x-y||/σ)³)³

triweight

Triweight kernel: k(x, y) = max(0, 1 - ||x-y||² / σ²)³