Expand description
Harmonic fits for making attention heads legible on chart coordinates.
The QK part is fit two ways:
- a stationary circulant kernel depending only on
t_q - t_k; - a separable low-harmonic surface on
(t_q, t_k)for heads whose score is not well described by phase difference alone.
Both fits are ordinary least squares in a fixed harmonic basis. The module does not choose harmonics by search; callers provide the maximum harmonic they want to inspect.
Structs§
- Attention
Kernel Fit - Attention
Kernel Report - Coordinate
MapFit - Harmonic
Basis Term - Harmonic
Coefficient - Harmonic
Content - Separable
Kernel Fit - Stationary
Kernel Fit