Struct SamplingParams 

pub struct SamplingParams {

    pub temperature: f32,
    pub top_p: f32,
    pub min_p: f32,
    pub top_k: usize,
    pub typical_p: f32,
    pub top_n_sigma: f32,
    pub xtc_probability: f32,
    pub xtc_threshold: f32,
    pub dry: DryParams,
    pub repetition_penalty: f32,
    pub penalty_last_n: usize,
    pub presence_penalty: f32,
    pub frequency_penalty: f32,
    pub sampler_order: SamplerOrder,
}

Sampling parameters for one generation request. temperature <= 0.0 means “sample nothing, take the greedy argmax” – the same deterministic behavior ferrox always had before this module existed.

Fields

temperature: f32

top_p: f32

Nucleus sampling threshold in (0.0, 1.0]. 1.0 disables top-p filtering (every token with nonzero probability is eligible).

min_p: f32

Keep only candidates at least min_p times as likely as the most likely one. 0.0 disables it; llama.cpp’s --min-p, whose default is 0.05 (common/common.h:231) rather than off.

That default is why this is a parity item and not a feature: llama.cpp truncates with min-p on every run nobody configured, so without it ferrox could not reproduce llama.cpp’s own out-of-the-box output for any prompt.

The struct default here stays 0.0 (disabled) for the same reason temperature defaults to greedy: SamplingParams::default is ferrox’s “do nothing the caller did not ask for” baseline, and llama.cpp’s CLI numbers live on the CLI flags.

top_k: usize

Keep only the top_k highest-probability tokens before sampling. 0 disables top-k filtering.

typical_p: f32

Locally typical sampling, llama.cpp’s typ_p (common/common.h:230, default 1.0 = disabled).

Keeps the candidates whose surprisal is CLOSEST to the distribution’s entropy, from the middle outward, rather than the most likely ones – see [crate::sampler_chain::Candidates::typical_p].

top_n_sigma: f32

Truncate at n standard deviations of the logits below the maximum, llama.cpp’s top_n_sigma (common/common.h:250, default -1.0 = disabled).

xtc_probability: f32

The probability that XTC removes the top candidates on any one token, llama.cpp’s xtc_probability (common/common.h:228, default 0.0 = disabled).

xtc_threshold: f32

The probability a candidate must reach to be a candidate XTC might remove, llama.cpp’s xtc_threshold (common/common.h:229, default 0.1). Above 0.5 disables XTC, which is upstream’s guard and not a range check: above 0.5 at most one candidate can ever clear it, and XTC never removes the last one.

dry: DryParams

The DRY sequence-repetition penalty. Disabled by default; see crate::dry for why its breakers are a type invariant rather than four more f32s here.

repetition_penalty: f32

1.0 discourages repeating a token already in the crate::penalty_window::PenaltyWindow – prompt included; 1.0 disables repetition penalty. Uses the standard convention (divide positive logits, multiply negative ones) so the penalty always pushes toward less likely, regardless of logit sign.

penalty_last_n: usize

How many of the most recent tokens the penalties look at, as llama.cpp’s penalty_last_n (common/common.h:238, default 64).

0 disables the penalties entirely. ferrox had no window at all and scanned the WHOLE history, so on a long generation it penalised a steadily growing set of tokens where llama.cpp penalises the last 64 – the divergence grew with output length, which is exactly when a repetition penalty matters most.

presence_penalty: f32

OpenAI-style presence penalty: subtract from logits of tokens that already appeared in the window (once per distinct token).

frequency_penalty: f32

OpenAI-style frequency penalty: subtract frequency_penalty * count from logits for each token id seen in the window.

sampler_order: SamplerOrder

The ORDER the chain above runs in, llama.cpp’s --samplers.

Not a cosmetic setting. Each filter renormalises over the survivors of the last one, so moving a step changes which candidates the next step can see – ferrox has already shipped that bug once, with temperature running first.

The default is llama.cpp’s own default chain (penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature), and every step ferrox added to it is a no-op at the neutral values above. See crate::sampler_order.

Implementations

impl SamplingParams

pub fn xtc_can_fire(&self) -> bool

The single predicate for “XTC can remove something”.

llama.cpp tests the same two conditions in two places – llama_sampler_init_xtc returns an empty sampler at :2208 and llama_sample_xtc_apply returns early at :2139 – and this is one function because ferrox reads it in two places too: the RNG draw (super::Sampler::xtc_roll) and the filter itself. If those disagreed, either the seeded stream would advance on a run XTC never touched (making an existing generation irreproducible) or XTC would ask for a draw nobody made.

pub fn greedy_equals_argmax(&self) -> bool

True when no step in this chain can move the argmax, so greedy decoding may skip building the candidate list entirely – and a backend may fold lm_head + argmax into its decode stack.

llama.cpp does not special-case temp <= 0: it runs the whole chain and lets the temperature step set every logit but the maximum to -inf (src/llama-sampler.cpp:271-286), so a filter that removed the maximum changes greedy output. Exactly two do:

• xtc removes the TOP candidates, by construction;
• typ_p selects outward from the distribution’s entropy and can drop the most likely token – llama.cpp’s own test case test_typical({0.4, 0.2, 0.2, 0.2}, {0.2, 0.2, 0.2}, 0.5) (tests/test-sampling.cpp:346) drops it.

dry changes logits rather than removing candidates, but it can change WHICH logit is the maximum, so it counts too. top_k, top_p, min_p and top_n_sigma all keep the maximum by construction, and penalties is applied to the whole vocabulary before the candidate list exists.

One predicate, three readers, because the alternative is this repo’s dominant defect: the sampler’s own greedy shortcut ([super::greedy_choice]), the Metal lm_head + argmax fold in ferrox_server::generate and the same fold in ferrox_cli::run must agree about it. A fold that ran while xtc was configured would hand the sampler a single precomputed id with no vocabulary left to remove anything from, and XTC would silently not run.

Chain membership is checked, not just the knob: a caller who set xtc_probability but left xtc out of --samplers asked for no XTC, and must keep the fast path.

Trait Implementations

impl Clone for SamplingParams

fn clone(&self) -> SamplingParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SamplingParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SamplingParams

fn default() -> Self

Greedy decoding: identical behavior to ferrox’s original argmax-only generation loop.