Struct EvaluationResult

pub struct EvaluationResult {
    pub log_probability: Option<f64>,
    pub log_perplexity: Option<f64>,
    pub log_perplexity_per_token: Option<f64>,
    pub log_perplexity_per_character: Option<f64>,
    pub correct_greedy: Option<bool>,
    pub token_count: Option<i32>,
    pub character_count: Option<i32>,
    pub completion: Option<String>,
}

Fields

log_probability: Option<f64>

log probability of producing the expected completion given the prompt. This metric refers to all tokens and is therefore dependent on the used tokenizer. It cannot be directly compared among models with different tokenizers.

log_perplexity: Option<f64>

log perplexity associated with the expected completion given the prompt. This metric refers to all tokens and is therefore dependent on the used tokenizer. It cannot be directly compared among models with different tokenizers.

log_perplexity_per_token: Option<f64>

log perplexity associated with the expected completion given the prompt normalized for the number of tokens. This metric computes an average per token and is therefore dependent on the used tokenizer. It cannot be directly compared among models with different tokenizers.

log_perplexity_per_character: Option<f64>

log perplexity associated with the expected completion given the prompt normalized for the number of characters. This metric is independent of any tokenizer. It can be directly compared among models with different tokenizers.

correct_greedy: Option<bool>

Flag indicating whether a greedy completion would have produced the expected completion.

token_count: Option<i32>

Number of tokens in the expected completion.

character_count: Option<i32>

Number of characters in the expected completion.

completion: Option<String>

argmax completion given the input consisting of prompt and expected completion. This may be used as an indicator of what the model would have produced. As only one single forward is performed an incoherent text could be produced especially for long expected completions.

Trait Implementations

impl Debug for EvaluationResult

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for EvaluationResult

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.