Struct OutputRequest

pub struct OutputRequest {
    pub all_logits: Option<Vec<f32>>,
    pub embeddings: Option<Vec<f32>>,
}

Used in a call to Model::evaluate or InferenceSession::infer to request information from the model. If a value is set to Some, the Vec will be cleared, resized, and filled with the related data.

Fields

all_logits: Option<Vec<f32>>

Returns all the logits for evaluation. A logit represents the likelihood that a given token will be generated based on the tokens that have been evaluated or generated so far. Output shape is n_batch * n_vocab.

embeddings: Option<Vec<f32>>

Returns all the embeddings for an evaluation. An embedding is a vector that measures the relatedness of text strings. Output shape is n_batch * n_embd.

Trait Implementations

impl Clone for OutputRequest

fn clone(&self) -> OutputRequest

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for OutputRequest

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for OutputRequest

fn default() -> OutputRequest

Returns the “default value” for a type.

impl PartialEq for OutputRequest

fn eq(&self, other: &OutputRequest) -> bool

Tests for self and other values to be equal, and is used by ==.

const fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for OutputRequest