Skip to main content

yarn_correction_range

Function yarn_correction_range 

Source
pub fn yarn_correction_range(
    scaling: YarnScaling,
    rotary_dim: usize,
    base: f32,
) -> (f64, f64)
Expand description

The [low, high] band range the YaRN ramp interpolates across, as the reference computes it (rotary.py:167-179): both ends from [yarn_correction_dim], floored / ceiled when truncate, low clamped up to 0, and – the load-bearing detail, called out in the reference’s own comment at rotary.py:176high clamped to rotary_dim - 1, not rotary_dim / 2 - 1.

The ramp only has rotary_dim / 2 entries, so a high above rotary_dim / 2 - 1 means the ramp never reaches 1.0: the longest-wavelength dims stay partly extrapolated. Clamping to the ramp’s own last index instead (the naive reading) forces the ramp to hit 1.0 at the last band and fully interpolates dims the reference deliberately leaves partly extrapolated – a checkpoint-wide change to the lowest frequencies, i.e. exactly the dims long-context behaviour rides on. Pinned by yarn_high_is_clamped_to_rotary_dim_minus_one_not_half_minus_one.

Returned as f64 because high may be fractional: when the range collapses (low == high, which is what truncate: false with beta_fast == beta_slow produces) the reference nudges high by +0.001 rather than flooring the gap at 1, and that nudge is what makes the ramp a step at low instead of a division by zero.