Struct aleph_alpha_client::Sampling

pub struct Sampling<'a> {
    pub temperature: Option<f64>,
    pub top_k: Option<u32>,
    pub top_p: Option<f64>,
    pub start_with_one_of: &'a [&'a str],
}

Sampling controls how the tokens (“words”) are selected for the completion.

Fields

temperature: Option<f64>

A temperature encourages teh model to produce less probable outputs (“be more creative”). Values are expected to be between 0 and 1. Try high values for a more random (“creative”) response.

top_k: Option<u32>

Introduces random sampling for generated tokens by randomly selecting the next token from the k most likely options. A value larger than 1 encourages the model to be more creative. Set to 0 to get the same behaviour as None.

top_p: Option<f64>

Introduces random sampling for generated tokens by randomly selecting the next token from the smallest possible set of tokens whose cumulative probability exceeds the probability top_p. Set to 0 to get the same behaviour as None.

start_with_one_of: &'a [&'a str]

Only start the completion with one of the following strings. The model will sample between these options, and ignore anything else. Once one of the options is generated, then the model will continue sampling until one of the stop reasons is reached.

For example, if trying to get the model to answer “Yes” or “No”, and your prompt was “Can this question be answered?” this could be set to [" Yes", " No"]. Note the space in front of each option, since the model would start with a space character.

Implementations

impl Sampling<'_>

pub const MOST_LIKELY: Self = _

Always chooses the token most likely to come next. Choose this if you do want close to deterministic behaviour and do not want to apply any penalties to avoid repetitions.

Trait Implementations

impl Default for Sampling<'_>

fn default() -> Self

Returns the “default value” for a type.