Struct llm_base::InferenceStats

pub struct InferenceStats {
    pub feed_prompt_duration: Duration,
    pub prompt_tokens: usize,
    pub predict_duration: Duration,
    pub predict_tokens: usize,
}

Statistics about the inference process.

Fields

feed_prompt_duration: Duration

How long it took to feed the prompt.

prompt_tokens: usize

How many tokens the prompt was.

predict_duration: Duration

How long it took to predict new tokens.

predict_tokens: usize

The number of predicted tokens.

Trait Implementations

impl Clone for InferenceStats

fn clone(&self) -> InferenceStats

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for InferenceStats

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for InferenceStats

fn default() -> Self

Returns the “default value” for a type.

impl Display for InferenceStats

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Copy for InferenceStats