Struct GenerationParams 

pub struct GenerationParams {
    pub temperature: f64,
    pub top_p: Option<f64>,
    pub top_k: Option<usize>,
    pub max_tokens: usize,
    pub seed: u64,
    pub repeat_penalty: f32,
    pub repeat_last_n: usize,
}

Sampling / generation parameters for Candle local inference.

Used inside [llm.candle.generation] or a [[llm.providers]] Candle entry.

Fields

temperature: f64

Sampling temperature. Higher values produce more creative outputs. Default: 0.7.

top_p: Option<f64>

Nucleus sampling threshold. When set, tokens with cumulative probability above this value are excluded. Default: None (disabled).

top_k: Option<usize>

Top-k sampling. When set, only the top-k most probable tokens are considered. Default: None (disabled).

max_tokens: usize

Maximum number of tokens to generate per response. Capped at MAX_TOKENS_CAP. Default: 2048.

seed: u64

Random seed for reproducible outputs. Default: 42.

repeat_penalty: f32

Repetition penalty applied during sampling. Default: 1.1.

repeat_last_n: usize

Number of last tokens to consider for the repetition penalty window. Default: 64.

Implementations

impl GenerationParams

pub fn capped_max_tokens(&self) -> usize

Returns max_tokens clamped to MAX_TOKENS_CAP.

Examples

use zeph_config::GenerationParams;

let params = GenerationParams::default();
assert!(params.capped_max_tokens() <= 32768);

Trait Implementations

impl Clone for GenerationParams

fn clone(&self) -> GenerationParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GenerationParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for GenerationParams

fn default() -> GenerationParams

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for GenerationParams

fn deserialize<__D>( __deserializer: __D, ) -> Result<GenerationParams, <__D as Deserializer<'de>>::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for GenerationParams

fn serialize<__S>( &self, __serializer: __S, ) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.