Struct SamplingParams 

pub struct SamplingParams {

    pub temperature: f32,
    pub top_p: f32,
    pub min_p: f32,
    pub top_k: usize,
    pub typical_p: f32,
    pub top_n_sigma: f32,
    pub xtc_probability: f32,
    pub xtc_threshold: f32,
    pub dry: DryParams,
    pub repetition_penalty: f32,
    pub penalty_last_n: usize,
    pub presence_penalty: f32,
    pub frequency_penalty: f32,
    pub sampler_order: SamplerOrder,
}

Sampling parameters for one generation request. temperature <= 0.0 means “sample nothing, take the greedy argmax” – the same deterministic behavior ferrox always had before this module existed.

Fields

temperature: f32

top_p: f32

Nucleus sampling threshold in (0.0, 1.0]. 1.0 disables top-p filtering (every token with nonzero probability is eligible).

min_p: f32

Keep only candidates at least min_p times as likely as the most likely one. 0.0 disables it; llama.cpp’s --min-p, whose default is 0.05 (common/common.h:231) rather than off.

That default is why this is a parity item and not a feature: llama.cpp truncates with min-p on every run nobody configured, so without it ferrox could not reproduce llama.cpp’s own out-of-the-box output for any prompt.

The struct default here stays 0.0 (disabled) for the same reason temperature defaults to greedy: SamplingParams::default is ferrox’s “do nothing the caller did not ask for” baseline, and llama.cpp’s CLI numbers live on the CLI flags.

top_k: usize

Keep only the top_k highest-probability tokens before sampling. 0 disables top-k filtering.

typical_p: f32

Locally typical sampling, llama.cpp’s typ_p (common/common.h:230, default 1.0 = disabled).

Keeps the candidates whose surprisal is CLOSEST to the distribution’s entropy, from the middle outward, rather than the most likely ones – see [crate::sampler_chain::Candidates::typical_p].

top_n_sigma: f32

Truncate at n standard deviations of the logits below the maximum, llama.cpp’s top_n_sigma (common/common.h:250, default -1.0 = disabled).

xtc_probability: f32

The probability that XTC removes the top candidates on any one token, llama.cpp’s xtc_probability (common/common.h:228, default 0.0 = disabled).

xtc_threshold: f32

The probability a candidate must reach to be a candidate XTC might remove, llama.cpp’s xtc_threshold (common/common.h:229, default 0.1). Above 0.5 disables XTC, which is upstream’s guard and not a range check: above 0.5 at most one candidate can ever clear it, and XTC never removes the last one.

dry: DryParams

The DRY sequence-repetition penalty. Disabled by default; see crate::dry for why its breakers are a type invariant rather than four more f32s here.

repetition_penalty: f32

1.0 discourages repeating a token already in the crate::penalty_window::PenaltyWindow – prompt included; 1.0 disables repetition penalty. Uses the standard convention (divide positive logits, multiply negative ones) so the penalty always pushes toward less likely, regardless of logit sign.

penalty_last_n: usize

How many of the most recent tokens the penalties look at, as llama.cpp’s penalty_last_n (common/common.h:238, default 64).

0 disables the penalties entirely. ferrox had no window at all and scanned the WHOLE history, so on a long generation it penalised a steadily growing set of tokens where llama.cpp penalises the last 64 – the divergence grew with output length, which is exactly when a repetition penalty matters most.

presence_penalty: f32

OpenAI-style presence penalty: subtract from logits of tokens that already appeared in the window (once per distinct token).

frequency_penalty: f32

OpenAI-style frequency penalty: subtract frequency_penalty * count from logits for each token id seen in the window.

sampler_order: SamplerOrder

The ORDER the chain above runs in, llama.cpp’s --samplers.

Not a cosmetic setting. Each filter renormalises over the survivors of the last one, so moving a step changes which candidates the next step can see – ferrox has already shipped that bug once, with temperature running first.

The default is llama.cpp’s own default chain (penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature), and every step ferrox added to it is a no-op at the neutral values above. See crate::sampler_order.

Implementations

impl SamplingParams

pub fn chain_keeps_the_argmax(&self) -> bool

True when no step LEFT in the chain can move the argmax of the scores it is handed – scores the penalties have ALREADY been applied to.

This is [super::greedy_choice]’s question and only its question. It is called per token, so it is a walk over at most crate::sampler_order::SamplerName::ALL.len() steps with no allocation, not a candidate list.

A device fold must ask Self::greedy_equals_raw_argmax instead: this one excuses the penalties because its caller already ran them, and a device that argmaxes raw logits has not.

pub fn greedy_equals_raw_argmax(&self) -> bool

True when the argmax of the raw logits is the token the whole sampler chain would choose, penalties included.

This is the question a backend folding lm_head + argmax into its decode stack has to ask, because the fold returns one token id and the host never sees a vocabulary: every step of the chain is skipped, not just the candidate-list filters.

Three readers, because the alternative is this repo’s dominant defect: ferrox_cli::run’s needs_vocab_logits, ferrox_server::generate’s, and through them the Metal fold’s own guard in ferrox_metal::greedy_fold.

Note what this costs: with the CLI’s default --repeat-penalty 1.1 the answer is false, so the Metal fold does not fire in the default configuration. That is deliberate – see issue #170 and this module’s header for the numbers. --repeat-penalty 1.0 or --repeat-last-n 0 gets it back.

impl SamplingParams

pub fn xtc_can_fire(&self) -> bool

The single predicate for “XTC can remove something”.

llama.cpp tests the same two conditions in two places – llama_sampler_init_xtc returns an empty sampler at :2208 and llama_sample_xtc_apply returns early at :2139 – and this is one function because ferrox reads it in two places too: the RNG draw (super::Sampler::xtc_roll) and the filter itself. If those disagreed, either the seeded stream would advance on a run XTC never touched (making an existing generation irreproducible) or XTC would ask for a draw nobody made.

Trait Implementations

impl Clone for SamplingParams

fn clone(&self) -> SamplingParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SamplingParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SamplingParams

fn default() -> Self

Greedy decoding: identical behavior to ferrox’s original argmax-only generation loop.