Expand description
Primitives shared by more than one model generation.
A function lives here only when every caller uses it with identical semantics and it bakes in no geometry. Anything conditional on generation stays with that generation.
Constants§
- EPS
- Epsilon used by both
_rms_unitand_zcrms, inside the square root. - SINKHORN_
ITERS - Sinkhorn iterations upstream runs.
Functions§
- rms_
unit decode._rms_unit— RMS normalisation with no learned scale.- rms_
unit_ to rms_unitinto a separate destination.- sinkhorn
architecture._sinkhorn— alternating log-domain row/column normalisation of ann x nmatrix, returningexp(log_K).