Struct StopConditions

pub struct StopConditions {
    pub max_tokens: Option<u32>,
    pub stop: Option<Vec<String>>,
    pub stop_token_ids_hidden: Option<Vec<TokenIdType>>,
    pub min_tokens: Option<u32>,
    pub ignore_eos: Option<bool>,
}

TensorRT LLM server-side stop conditions. These options allow for the server to evaluate the generated sequence and stop generation if the sequence meets a stop condition.

Fields

max_tokens: Option<u32>

The maximum number of tokens to generate

stop: Option<Vec<String>>

List of strings that stop the generation when they are generated. The returned output will not contain the stop strings.

stop_token_ids_hidden: Option<Vec<TokenIdType>>

List of tokens that stop the generation when they are generated. The returned output will NOT contain the stop tokens.

min_tokens: Option<u32>

The minimum number of tokens to generate To ignore_eos, set min_tokens to max_tokens

ignore_eos: Option<bool>

Whether to ignore the EOS token and continue generating tokens after the EOS token is generated.

Implementations

impl StopConditions

pub fn apply_ignore_eos(&mut self)

Trait Implementations

impl Clone for StopConditions

fn clone(&self) -> StopConditions

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for StopConditions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for StopConditions

fn default() -> StopConditions

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for StopConditions

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for StopConditions

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.