Struct aleph_alpha_client::Sampling

pub struct Sampling {
    pub temperature: Option<f64>,
    pub top_k: Option<u32>,
    pub top_p: Option<f64>,
}

Sampling controls how the tokens (“words”) are selected for the completion.

Fields

temperature: Option<f64>

A temperature encourages teh model to produce less probable outputs (“be more creative”). Values are expected to be between 0 and 1. Try high values for a more random (“creative”) response.

top_k: Option<u32>

Introduces random sampling for generated tokens by randomly selecting the next token from the k most likely options. A value larger than 1 encourages the model to be more creative. Set to 0 to get the same behaviour as None.

top_p: Option<f64>

Introduces random sampling for generated tokens by randomly selecting the next token from the smallest possible set of tokens whose cumulative probability exceeds the probability top_p. Set to 0 to get the same behaviour as None.

Implementations

impl Sampling

pub const MOST_LIKELY: Self = _

Always chooses the token most likely to come next. Choose this if you do want close to deterministic behaviour and do not want to apply any penalties to avoid repetitions.