pub fn yarn_correction_range(
scaling: YarnScaling,
rotary_dim: usize,
base: f32,
) -> (f64, f64)Expand description
The [low, high] band range the YaRN ramp interpolates across, as
the reference computes it (rotary.py:167-179): both ends from
[yarn_correction_dim], floored / ceiled when truncate, low
clamped up to 0, and – the load-bearing detail, called out in the
reference’s own comment at rotary.py:176 – high clamped to
rotary_dim - 1, not rotary_dim / 2 - 1.
The ramp only has rotary_dim / 2 entries, so a high above
rotary_dim / 2 - 1 means the ramp never reaches 1.0: the
longest-wavelength dims stay partly extrapolated. Clamping to the
ramp’s own last index instead (the naive reading) forces the ramp to
hit 1.0 at the last band and fully interpolates dims the reference
deliberately leaves partly extrapolated – a checkpoint-wide change
to the lowest frequencies, i.e. exactly the dims long-context
behaviour rides on. Pinned by
yarn_high_is_clamped_to_rotary_dim_minus_one_not_half_minus_one.
Returned as f64 because high may be fractional: when the range
collapses (low == high, which is what truncate: false with
beta_fast == beta_slow produces) the reference nudges high by
+0.001 rather than flooring the gap at 1, and that nudge is what
makes the ramp a step at low instead of a division by zero.