Struct TextInferenceConfigBuilder 

#[non_exhaustive]
pub struct TextInferenceConfigBuilder { /* private fields */ }

A builder for TextInferenceConfig.

Implementations

impl TextInferenceConfigBuilder

pub fn temperature(self, input: f32) -> Self

Controls the random-ness of text generated by the language model, influencing how much the model sticks to the most predictable next words versus exploring more surprising options. A lower temperature value (e.g. 0.2 or 0.3) makes model outputs more deterministic or predictable, while a higher temperature (e.g. 0.8 or 0.9) makes the outputs more creative or unpredictable.

pub fn set_temperature(self, input: Option<f32>) -> Self

Controls the random-ness of text generated by the language model, influencing how much the model sticks to the most predictable next words versus exploring more surprising options. A lower temperature value (e.g. 0.2 or 0.3) makes model outputs more deterministic or predictable, while a higher temperature (e.g. 0.8 or 0.9) makes the outputs more creative or unpredictable.

pub fn get_temperature(&self) -> &Option<f32>

Controls the random-ness of text generated by the language model, influencing how much the model sticks to the most predictable next words versus exploring more surprising options. A lower temperature value (e.g. 0.2 or 0.3) makes model outputs more deterministic or predictable, while a higher temperature (e.g. 0.8 or 0.9) makes the outputs more creative or unpredictable.

pub fn top_p(self, input: f32) -> Self

A probability distribution threshold which controls what the model considers for the set of possible next tokens. The model will only consider the top p% of the probability distribution when generating the next token.

pub fn set_top_p(self, input: Option<f32>) -> Self

A probability distribution threshold which controls what the model considers for the set of possible next tokens. The model will only consider the top p% of the probability distribution when generating the next token.

pub fn get_top_p(&self) -> &Option<f32>

A probability distribution threshold which controls what the model considers for the set of possible next tokens. The model will only consider the top p% of the probability distribution when generating the next token.

pub fn max_tokens(self, input: i32) -> Self

The maximum number of tokens to generate in the output text. Do not use the minimum of 0 or the maximum of 65536. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.

pub fn set_max_tokens(self, input: Option<i32>) -> Self

The maximum number of tokens to generate in the output text. Do not use the minimum of 0 or the maximum of 65536. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.

pub fn get_max_tokens(&self) -> &Option<i32>

The maximum number of tokens to generate in the output text. Do not use the minimum of 0 or the maximum of 65536. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.

pub fn stop_sequences(self, input: impl Into<String>) -> Self

Appends an item to stop_sequences.

To override the contents of this collection use set_stop_sequences.

A list of sequences of characters that, if generated, will cause the model to stop generating further tokens. Do not use a minimum length of 1 or a maximum length of 1000. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.

pub fn set_stop_sequences(self, input: Option<Vec<String>>) -> Self

A list of sequences of characters that, if generated, will cause the model to stop generating further tokens. Do not use a minimum length of 1 or a maximum length of 1000. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.

pub fn get_stop_sequences(&self) -> &Option<Vec<String>>

A list of sequences of characters that, if generated, will cause the model to stop generating further tokens. Do not use a minimum length of 1 or a maximum length of 1000. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.

pub fn build(self) -> TextInferenceConfig

Consumes the builder and constructs a TextInferenceConfig.

Trait Implementations

impl Clone for TextInferenceConfigBuilder

fn clone(&self) -> TextInferenceConfigBuilder

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TextInferenceConfigBuilder

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TextInferenceConfigBuilder

fn default() -> TextInferenceConfigBuilder

Returns the “default value” for a type.

impl PartialEq for TextInferenceConfigBuilder

fn eq(&self, other: &TextInferenceConfigBuilder) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for TextInferenceConfigBuilder