Module alibi 

ALiBi – the per-head linear position bias llama.cpp adds inside ggml_soft_max_ext when hparams.f_max_alibi_bias > 0.

What it is

ggml_soft_max_ext(kq, kq_mask, scale, max_bias) computes softmax(scale * kq + slope_h * mask), where the mask a model with use_alibi fills is -|p_key - p_query| on the visible entries and -inf elsewhere (llama-kv-cache.cpp:1673-1676), and the slope is ggml-cpu/ops.cpp:5489-5508:

n_head_log2 = 2^floor(log2(n_head))
m0 = 2^(-max_bias / n_head_log2)
m1 = 2^(-(max_bias / 2) / n_head_log2)
slope_h = h < n_head_log2 ? m0^(h + 1) : m1^(2 (h - n_head_log2) + 1)

So a query at p_q sees key p_k <= p_q with slope_h * (p_k - p_q) added to its scaled score, after any softcap (the softcapped graphs scale and tanh BEFORE ggml_soft_max_ext; none of them uses ALiBi, but the order is the graph’s). slopes is the formula; the kernels in attention take the result as an optional per-head slice and rotate nothing for such a model.

The graphs that set f_max_alibi_bias are the caller’s business (ferrox_models::alibi); this module is the arithmetic.

Functions

slopes

One slope per query head, n_heads long, for a max_bias that is positive. Returns None for a non-positive max_bias, which is llama.cpp’s “no ALiBi” (slope = 1.0 on a mask that is then 0 / -inf, i.e. no bias at all).