Struct PromptModelInferenceConfiguration 

#[non_exhaustive]
pub struct PromptModelInferenceConfiguration {
    pub temperature: Option<f32>,
    pub top_p: Option<f32>,
    pub max_tokens: Option<i32>,
    pub stop_sequences: Option<Vec<String>>,
}

Contains inference configurations related to model inference for a prompt. For more information, see Inference parameters.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

temperature: Option<f32>

Controls the randomness of the response. Choose a lower value for more predictable outputs and a higher value for more surprising outputs.

top_p: Option<f32>

The percentage of most-likely candidates that the model considers for the next token.

max_tokens: Option<i32>

The maximum number of tokens to return in the response.

stop_sequences: Option<Vec<String>>

A list of strings that define sequences after which the model will stop generating.

Implementations

impl PromptModelInferenceConfiguration

pub fn temperature(&self) -> Option<f32>

Controls the randomness of the response. Choose a lower value for more predictable outputs and a higher value for more surprising outputs.

pub fn top_p(&self) -> Option<f32>

The percentage of most-likely candidates that the model considers for the next token.

pub fn max_tokens(&self) -> Option<i32>

The maximum number of tokens to return in the response.

pub fn stop_sequences(&self) -> &[String]

A list of strings that define sequences after which the model will stop generating.

If no value was sent for this field, a default will be set. If you want to determine if no value was sent, use .stop_sequences.is_none().

impl PromptModelInferenceConfiguration

pub fn builder() -> PromptModelInferenceConfigurationBuilder

Creates a new builder-style object to manufacture PromptModelInferenceConfiguration.

Trait Implementations

impl Clone for PromptModelInferenceConfiguration

fn clone(&self) -> PromptModelInferenceConfiguration

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PromptModelInferenceConfiguration

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for PromptModelInferenceConfiguration

fn eq(&self, other: &PromptModelInferenceConfiguration) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for PromptModelInferenceConfiguration