Struct Stopping

pub struct Stopping<'a> {
    pub maximum_tokens: Option<u32>,
    pub stop_sequences: &'a [&'a str],
}

Controls the conditions under which the language models stops generating text.

Fields

maximum_tokens: Option<u32>

The maximum number of tokens to be generated. Completion will terminate after the maximum number of tokens is reached. Increase this value to allow for longer outputs. A text is split into tokens. Usually there are more tokens than words. The total number of tokens of prompt and maximum_tokens depends on the model. If maximum tokens is set to None, no outside limit is opposed on the number of maximum tokens. The model will generate tokens until it generates one of the specified stop_sequences or it reaches its technical limit, which usually is its context window.

stop_sequences: &'a [&'a str]

List of strings which will stop generation if they are generated. Stop sequences are helpful in structured texts. E.g.: In a question answering scenario a text may consist of lines starting with either “Question: “ or “Answer: “ (alternating). After producing an answer, the model will be likely to generate “Question: “. “Question: “ may therefore be used as stop sequence in order not to have the model generate more questions but rather restrict text generation to the answers.

Implementations

impl<'a> Stopping<'a>

pub const NO_TOKEN_LIMIT: Self

Only stop once the model reaches its technical limit, usually the context window.

pub fn from_maximum_tokens(maximum_tokens: u32) -> Self

Stop once the model has reached maximum_tokens.

pub fn from_stop_sequences(stop_sequences: &'a [&'a str]) -> Self

Trait Implementations

impl Default for Stopping<'_>

fn default() -> Self

Returns the “default value” for a type.