Module kernel_alignment 

Kernel Alignment

Implements Kernel Target Alignment (KTA), Centered Kernel Alignment (CKA), HSIC (Hilbert-Schmidt Independence Criterion), and optimization routines for selecting kernel hyperparameters based on alignment with a target kernel.

Overview

Kernel alignment metrics quantify how well a kernel matrix captures the structure of the learning problem. Given labels, one constructs an “ideal” target kernel where T[i,j] = +1 if labels[i] == labels[j] and -1 otherwise. A high alignment between K and T indicates that the kernel maps similar-class points close together.

Kernel Target Alignment (KTA)

KTA(K, T) = <K, T>_F / (||K||_F * ||T||_F)

Centered Kernel Alignment (CKA)

CKA applies double-centering before computing alignment, making it invariant to isotropic scaling and constant shifts:

CKA(K1, K2) = HSIC(K1, K2) / sqrt(HSIC(K1,K1) * HSIC(K2,K2))

where HSIC(K, L) = (1/n^2) * <H*K*H, H*L*H>_F.

References

• Cortes, C., Mohri, M., & Rostamizadeh, A. (2012). Algorithms for learning kernels based on centered alignment. JMLR.
• Kornblith, S., et al. (2019). Similarity of neural network representations revisited. ICML.

Structs

AlignmentOptConfig

Configuration for alignment-based kernel hyperparameter search.

AlignmentResult

Result of a pairwise kernel alignment computation.

AlignmentStats

Comprehensive alignment statistics between a kernel K and a target kernel.

KernelMatrix

A square kernel matrix (n×n, symmetric positive semi-definite).

OptimizationResult

Outcome of a kernel alignment optimisation run.

Enums

AlignmentError

Errors that can arise during kernel alignment computations.

Functions

alignment_stats

Compute all alignment metrics in a single pass.

centered_kernel_alignment

Compute the Centered Kernel Alignment (CKA) between kernel matrices k1 and k2.

gradient_ascent_alignment

Gradient ascent on the alignment score via finite differences.

grid_search_alignment

Grid search over a discrete set of kernel parameter vectors, returning the one that maximises alignment with target.

hsic

Compute the biased HSIC (Hilbert-Schmidt Independence Criterion) estimate:

kernel_target_alignment

Compute the Kernel Target Alignment (KTA) between kernel k and target kernel target.