pub enum O1Rect {
Aggregate,
Fm,
}Expand description
Which rectifier keeps the skeleton’s estimated far mass non-negative.
pinv(exp(Q̃K̃ᵀ/√d)) is violently ill-conditioned, so M is indefinite and the raw skeleton estimates negative weights for a large minority of keys (measured on Qwen3-0.6B: 23.5% of far weights negative, carrying 24.5% of the absolute far mass). Unrectified, the joint denominator goes near-zero/negative and the model collapses (×510).
The matrix probe rectifies every estimated weight — west = ((Fu@Mu)@E).clamp_min(0). A STREAMING kernel cannot do that: the
per-(t, j) weights are never materialized, they exist only already
contracted against the accumulators. Two streaming-legal stand-ins:
MEASURED (Qwen3-0.6B, all 28 layers, W=128, sink=4, wikitext-2 val,
12×512 windows, landmarks frozen at a 256-token prefill — i.e. the
runtime’s real discipline, cortiq ppl --o1):
m=8 m=16 m=32
agg 28.51 (×1.318) 28.82 (×1.332) 28.04 (×1.296) ← default
fm 28.97 (×1.340) 29.69 (×1.373) 30.58 (×1.414)Aggregate wins at every m, so it stays the default. Fm is kept
selectable because its per-key guarantee is the intuitively “correct”
fix and someone will re-derive it: this table is the evidence that it
costs quality HERE, and the reason is that the guarantee is bought by
destroying signal — clamping a landmark’s coefficient zeroes its
contribution to EVERY far key, including the majority where the
weighted sum was already positive and accurate.
Variants§
Aggregate
Clamp only the AGGREGATE far denominator: a row whose skeleton denominator comes out negative drops its far field entirely. Coarse — negative per-key mass survives untouched whenever the row sum happens to stay positive — but measured BEST (see above): the surviving negatives are apparently error-cancelling, not error-causing.
Fm
Clamp FM = F_u·M_u (an m-vector, per query row) at zero.
ŵ(t,j) = Σ_b FM[b]·E[b,j] and E = exp(·) ≥ 0 ELEMENTWISE, so
FM ≥ 0 is SUFFICIENT for every far weight to be non-negative —
a per-key guarantee bought with O(m) work on a vector the row
already materializes, state untouched. It is strictly stronger
than the probe’s clamp (a negative landmark is dropped for every
key, not only where the sum would go negative), so this is a
DIFFERENT operator, not an emulation of the matrix reference —
and, measured, a worse one. Opt in with --o1-rect fm.