Skip to main content

Module kernels

Module kernels 

Source
Expand description

Primitives shared by more than one model generation.

A function lives here only when every caller uses it with identical semantics and it bakes in no geometry. Anything conditional on generation stays with that generation.

Constants§

EPS
Epsilon used by both _rms_unit and _zcrms, inside the square root.
SINKHORN_ITERS
Sinkhorn iterations upstream runs.

Functions§

rms_unit
decode._rms_unit — RMS normalisation with no learned scale.
rms_unit_to
rms_unit into a separate destination.
sinkhorn
architecture._sinkhorn — alternating log-domain row/column normalisation of an n x n matrix, returning exp(log_K).