Struct rust_bert::pipelines::generation_utils::GenerateOptions [−][src]
pub struct GenerateOptions<'a> {Show 20 fields
pub min_length: Option<i64>,
pub max_length: Option<i64>,
pub max_new_tokens: Option<i64>,
pub early_stopping: Option<bool>,
pub num_return_sequences: Option<i64>,
pub num_beams: Option<i64>,
pub num_beam_groups: Option<i64>,
pub do_sample: Option<bool>,
pub temperature: Option<f64>,
pub top_k: Option<i64>,
pub top_p: Option<f64>,
pub repetition_penalty: Option<f64>,
pub length_penalty: Option<f64>,
pub no_repeat_ngram_size: Option<i64>,
pub diversity_penalty: Option<f64>,
pub decoder_start_token_id: Option<i64>,
pub forced_bos_token_id: Option<i64>,
pub prefix_allowed_tokens_fn: Option<&'a dyn Fn(i64, &Tensor) -> Vec<i64>>,
pub bad_word_ids: Option<&'a Vec<Vec<i64>>>,
pub output_scores: bool,
}
Expand description
Generation options for text generation.
When provided to a generate
method, these options will take priority over the GenerateConfig
used to create the
LanguageGenerator
. Some of these options may be left as None
, options without a value will individually default
to the GenerateConfig
.
Fields
min_length: Option<i64>
Minimum sequence length
max_length: Option<i64>
Maximum sequence length
max_new_tokens: Option<i64>
Maximum number of new tokens to generate (useful for causal generation models).
Only one of max_length
and max_new_tokens
should be provided.
When both are given, max_new_tokens
is ignored and the max_length
setting is used.
early_stopping: Option<bool>
Early stopping flag indicating if the beam search should stop as soon as num_beam
hypotheses have been generated
num_return_sequences: Option<i64>
Number of sequences to return for each prompt text
num_beams: Option<i64>
Number of beams for beam search
num_beam_groups: Option<i64>
do_sample: Option<bool>
Sampling flag. If true, will perform top-k and/or nucleus sampling on generated tokens, otherwise greedy (deterministic) decoding
temperature: Option<f64>
Temperature setting. Values higher than 1 will improve originality at the risk of reducing relevance
top_k: Option<i64>
Top_k values for sampling tokens. Value higher than 0 will enable the feature
top_p: Option<f64>
Top_p value for Nucleus sampling, Holtzman et al.. Keep top tokens until cumulative probability reaches top_p
repetition_penalty: Option<f64>
Repetition penalty (mostly useful for CTRL decoders). Values higher than 1 will penalize tokens that have been already generated.
length_penalty: Option<f64>
Exponential penalty based on the length of the hypotheses generated
no_repeat_ngram_size: Option<i64>
Number of allowed repetitions of n-grams. Values higher than 0 turn on this feature
diversity_penalty: Option<f64>
Diversity penalty for diverse beam search. High values will enforce more difference between beam groups
decoder_start_token_id: Option<i64>
Decoder start token id
forced_bos_token_id: Option<i64>
Forced first token generated
prefix_allowed_tokens_fn: Option<&'a dyn Fn(i64, &Tensor) -> Vec<i64>>
Function to control the generation process. The function should take a batch_id
(i64) and a tensor of token_ids already generated and returns a Vec<i64>
of allowed tokens.
bad_word_ids: Option<&'a Vec<Vec<i64>>>
List of bad word ids (may be a sequence of word ids) that will be banned during the generation
output_scores: bool
Flag indicating if text generation scores should be returned
Trait Implementations
Returns the “default value” for a type. Read more
Auto Trait Implementations
impl<'a> !RefUnwindSafe for GenerateOptions<'a>
impl<'a> !Send for GenerateOptions<'a>
impl<'a> !Sync for GenerateOptions<'a>
impl<'a> Unpin for GenerateOptions<'a>
impl<'a> !UnwindSafe for GenerateOptions<'a>
Blanket Implementations
Mutably borrows from an owned value. Read more
Instruments this type with the provided Span
, returning an
Instrumented
wrapper. Read more
type Output = T
type Output = T
Should always be Self