Struct llm_base::EvaluateOutputRequest

pub struct EvaluateOutputRequest {
    pub all_logits: Option<Vec<f32>>,
    pub embeddings: Option<Vec<f32>>,
}

Used in a call to Model::evaluate or InferenceSession::infer to request information from the model. If a value is set to Some, the Vec will be cleared, resized, and filled with the related data.

Fields

all_logits: Option<Vec<f32>>

Returns all the logits for evaluation. A logit represents the likelihood that a given token will be generated based on the tokens that have been evaluated or generated so far. Output shape is n_batch * n_vocab.

embeddings: Option<Vec<f32>>

Returns all the embeddings for an evaluation. An embedding is a vector that measures the relatedness of text strings. Output shape is n_batch * n_embd.

Trait Implementations

impl Clone for EvaluateOutputRequest

fn clone(&self) -> EvaluateOutputRequest

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EvaluateOutputRequest

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for EvaluateOutputRequest

fn default() -> EvaluateOutputRequest

Returns the “default value” for a type.