Struct Usage 

pub struct Usage {
    pub prompt_tokens: usize,
    pub completion_tokens: usize,
    pub total_tokens: usize,
    pub prompt_per_second: Option<f64>,
    pub predicted_per_second: Option<f64>,
    pub prompt_eval_duration_ms: Option<f64>,
    pub generation_duration_ms: Option<f64>,
    pub time_to_first_token_ms: Option<f64>,
    pub cached_tokens: Option<usize>,
}

Fields

prompt_tokens: usize

completion_tokens: usize

total_tokens: usize

prompt_per_second: Option<f64>

Prefill throughput (prompt tokens / prefill seconds), when timed.

predicted_per_second: Option<f64>

Decode throughput (completion tokens / decode seconds), when timed.

prompt_eval_duration_ms: Option<f64>

Wall time spent processing the prompt, in milliseconds.

generation_duration_ms: Option<f64>

Wall time spent in the decode loop, in milliseconds. Kept separate from prompt_eval_duration_ms on purpose (see the module docs).

time_to_first_token_ms: Option<f64>

Time to first token: from the start of prefill to the moment the first token was produced. None when no token was produced at all (an immediate EOS), because a zero there would read as an instantaneous response.

cached_tokens: Option<usize>

Prompt tokens served from the KV prefix cache instead of being recomputed. Some(0) means “the cache was consulted and missed”; None means “no prefix cache is configured” – a distinction the UI needs to decide whether to show the row at all.

Implementations

impl Usage

pub fn new(prompt_tokens: usize, completion_tokens: usize) -> Self

pub fn with_timings(self, prompt_secs: f64, predicted_secs: f64) -> Self

Records the two phase durations, in seconds, and the rates they imply. A zero-length phase leaves the rate unset rather than dividing by zero into infinity.

pub fn with_ttft(self, secs: f64) -> Self

Time-to-first-token, in seconds, measured from the start of prefill. Ignored when no token was generated.

pub fn with_cached_tokens(self, cached: usize) -> Self

Prompt tokens that came from the prefix cache. Call this only when a prefix cache actually exists (see cached_tokens).

Trait Implementations

impl Clone for Usage

fn clone(&self) -> Usage

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Usage

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for Usage

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for Usage

fn eq(&self, other: &Usage) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for Usage

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for Usage