Struct rust_bert::pipelines::text_generation::TextGenerationConfig

pub struct TextGenerationConfig {

    pub model_type: ModelType,
    pub model_resource: Resource,
    pub config_resource: Resource,
    pub vocab_resource: Resource,
    pub merges_resource: Resource,
    pub min_length: i64,
    pub max_length: i64,
    pub do_sample: bool,
    pub early_stopping: bool,
    pub num_beams: i64,
    pub temperature: f64,
    pub top_k: i64,
    pub top_p: f64,
    pub repetition_penalty: f64,
    pub length_penalty: f64,
    pub no_repeat_ngram_size: i64,
    pub num_return_sequences: i64,
    pub num_beam_groups: Option<i64>,
    pub diversity_penalty: Option<f64>,
    pub device: Device,
}

Configuration for text generation

Contains information regarding the model to load, mirrors the GenerateConfig, with a different set of default parameters and sets the device to place the model on.

Fields

model_type: ModelType

Model type

model_resource: Resource

Model weights resource (default: pretrained BART model on CNN-DM)

config_resource: Resource

Config resource (default: pretrained BART model on CNN-DM)

vocab_resource: Resource

Vocab resource (default: pretrained BART model on CNN-DM)

merges_resource: Resource

Merges resource (default: pretrained BART model on CNN-DM)

min_length: i64

Minimum sequence length (default: 0)

max_length: i64

Maximum sequence length (default: 20)

do_sample: bool

Sampling flag. If true, will perform top-k and/or nucleus sampling on generated tokens, otherwise greedy (deterministic) decoding (default: true)

early_stopping: bool

Early stopping flag indicating if the beam search should stop as soon as num_beam hypotheses have been generated (default: false)

num_beams: i64

Number of beams for beam search (default: 5)

temperature: f64

Temperature setting. Values higher than 1 will improve originality at the risk of reducing relevance (default: 1.0)

top_k: i64

Top_k values for sampling tokens. Value higher than 0 will enable the feature (default: 0)

top_p: f64

Top_p value for Nucleus sampling, Holtzman et al.. Keep top tokens until cumulative probability reaches top_p (default: 0.9)

repetition_penalty: f64

Repetition penalty (mostly useful for CTRL decoders). Values higher than 1 will penalize tokens that have been already generated. (default: 1.0)

length_penalty: f64

Exponential penalty based on the length of the hypotheses generated (default: 1.0)

no_repeat_ngram_size: i64

Number of allowed repetitions of n-grams. Values higher than 0 turn on this feature and will prevent repeats of n-grams with a length equal or greater to this value (default: 0)

num_return_sequences: i64

Number of sequences to return for each prompt text (default: 1)

num_beam_groups: Option<i64>

Number of beam groups for diverse beam generation. If provided and higher than 1, will split the beams into beam subgroups leading to more diverse generation.

diversity_penalty: Option<f64>

Diversity penalty for diverse beam search. High values will enforce more difference between beam groups (default: 5.5)

device: Device

Device to place the model on (default: CUDA/GPU when available)

Implementations

impl TextGenerationConfig

pub fn new(
    model_type: ModelType,
    model_resource: Resource,
    config_resource: Resource,
    vocab_resource: Resource,
    merges_resource: Resource
) -> TextGenerationConfig

Instantiate a new text generation configuration of the supplied type.

Arguments

• model_type - ModelType indicating the model type to load (must match with the actual data to be loaded!)
• model_resource - The Resource pointing to the model to load (e.g. model.ot)
• config_resource - The `Resource’ pointing to the model configuration to load (e.g. config.json)
• vocab_resource - The `Resource’ pointing to the tokenizer’s vocabulary to load (e.g. vocab.txt/vocab.json)
• merges_resource - The Resource pointing to the tokenizer’s merge file or SentencePiece model to load (e.g. merges.txt).

Trait Implementations

impl Default for TextGenerationConfig

fn default() -> TextGenerationConfig

Returns the “default value” for a type.

impl From<TextGenerationConfig> for GenerateConfig

fn from(config: TextGenerationConfig) -> GenerateConfig

Performs the conversion.