Struct SamplingParams 

#[non_exhaustive]
pub struct SamplingParams {
    pub top_p: f32,
    pub top_k: i32,
    pub min_p: f32,
    pub presence_penalty: f32,
    pub repetition_penalty: f32,
}

Sampling parameters that control token generation.

Marked #[non_exhaustive] so future sampling knobs can be added without a breaking release. Start from SamplingParams::default and chain with_* setters:

let params = rig_llama_cpp::SamplingParams::default()
    .with_top_k(40)
    .with_presence_penalty(1.5);

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

top_p: f32

Nucleus sampling threshold (default: 0.95).

top_k: i32

Top-k sampling parameter (default: 40).

min_p: f32

Minimum probability threshold (default: 0.0).

presence_penalty: f32

Penalty for token presence (default: 0.0).

repetition_penalty: f32

Penalty for token repetition (default: 1.0).

Implementations

impl SamplingParams

pub fn with_top_p(self, top_p: f32) -> Self

Set the nucleus sampling threshold.

pub fn with_top_k(self, top_k: i32) -> Self

Set the top-k sampling parameter.

pub fn with_min_p(self, min_p: f32) -> Self

Set the minimum probability threshold.

pub fn with_presence_penalty(self, presence_penalty: f32) -> Self

Set the presence penalty.

pub fn with_repetition_penalty(self, repetition_penalty: f32) -> Self

Set the repetition penalty.

Trait Implementations

impl Clone for SamplingParams

fn clone(&self) -> SamplingParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SamplingParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SamplingParams

fn default() -> Self

Returns the “default value” for a type.

impl Copy for SamplingParams