pub struct ChatSampling {
pub temperature: Option<f64>,
pub top_p: Option<f64>,
pub frequency_penalty: Option<f64>,
pub presence_penalty: Option<f64>,
}
Expand description
Sampling controls how the tokens (“words”) are selected for the completion. This is different
from crate::Sampling
, because it does not supprot the top_k
parameter.
Fields§
§temperature: Option<f64>
A temperature encourages the model to produce less probable outputs (“be more creative”). Values are expected to be between 0 and 1. Try high values for a more random (“creative”) response.
top_p: Option<f64>
Introduces random sampling for generated tokens by randomly selecting the next token from
the k most likely options. A value larger than 1 encourages the model to be more creative.
Set to 0 to get the same behaviour as None
.
frequency_penalty: Option<f64>
When specified, this number will decrease (or increase) the likelihood of repeating tokens that were mentioned prior in the completion. The penalty is cumulative. The more a token is mentioned in the completion, the more its probability will decrease. A negative value will increase the likelihood of repeating tokens.
presence_penalty: Option<f64>
The presence penalty reduces the likelihood of generating tokens that are already present in the generated text (repetition_penalties_include_completion=true) respectively the prompt (repetition_penalties_include_prompt=true). Presence penalty is independent of the number of occurrences. Increase the value to reduce the likelihood of repeating text. An operation like the following is applied:
logits[t] -> logits[t] - 1 * penalty
where logits[t] is the logits for any given token. Note that the formula is independent of the number of times that a token appears.
Implementations§
Source§impl ChatSampling
impl ChatSampling
Sourcepub const MOST_LIKELY: Self
pub const MOST_LIKELY: Self
Always chooses the token most likely to come next. Choose this if you do want close to deterministic behaviour and do not want to apply any penalties to avoid repetitions.
Trait Implementations§
Auto Trait Implementations§
impl Freeze for ChatSampling
impl RefUnwindSafe for ChatSampling
impl Send for ChatSampling
impl Sync for ChatSampling
impl Unpin for ChatSampling
impl UnwindSafe for ChatSampling
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> Pointable for T
impl<T> Pointable for T
Source§impl<T> PolicyExt for Twhere
T: ?Sized,
impl<T> PolicyExt for Twhere
T: ?Sized,
Source§impl<R, P> ReadPrimitive<R> for P
impl<R, P> ReadPrimitive<R> for P
Source§fn read_from_little_endian(read: &mut R) -> Result<Self, Error>
fn read_from_little_endian(read: &mut R) -> Result<Self, Error>
ReadEndian::read_from_little_endian()
.