Struct CompletionParams 

pub struct CompletionParams {
    pub max_tokens: Option<u32>,
    pub temperature: Option<f64>,
    pub top_k: Option<u32>,
    pub top_p: Option<f64>,
    pub stop: Vec<String>,
    pub return_special_tokens: bool,
    pub frequency_penalty: Option<f64>,
    pub presence_penalty: Option<f64>,
    pub logprobs: Logprobs,
}

Completion request parameters

Fields

max_tokens: Option<u32>

The maximum tokens that should be inferred.

Note: the backing implementation may return less tokens due to other stop reasons.

temperature: Option<f64>

The randomness with which the next token is selected.

top_k: Option<u32>

The number of possible next tokens the model will choose from.

top_p: Option<f64>

The probability total of next tokens the model will choose from.

stop: Vec<String>

A list of sequences that, if encountered, the API will stop generating further tokens.

return_special_tokens: bool

Whether to include special tokens like <|eot_id|> in the completion

frequency_penalty: Option<f64>

When specified, this number will decrease (or increase) the probability of repeating tokens that were mentioned prior in the completion. The penalty is cumulative. The more a token is mentioned in the completion, the more its probability will decrease. A negative value will increase the likelihood of repeating tokens.

presence_penalty: Option<f64>

The presence penalty reduces the probability of generating tokens that are already present in the generated text respectively prompt. Presence penalty is independent of the number of occurrences. Increase the value to reduce the probability of repeating text.

logprobs: Logprobs

Use this to control the logarithmic probabilities you want to have returned. This is useful to figure out how likely it had been that this specific token had been sampled.

Trait Implementations

impl Clone for CompletionParams

fn clone(&self) -> CompletionParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CompletionParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CompletionParams

fn default() -> Self

Returns the “default value” for a type.

impl Serialize for CompletionParams

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.