Struct StreamStats 

pub struct StreamStats {
    pub tokens_generated: usize,
    pub prefill_tokens: usize,
    pub time_to_first_token_ms: u64,
    pub total_time_ms: u64,
    pub tokens_per_second: f32,
}

Per-request generation throughput statistics.

Fields

tokens_generated: usize

Total tokens emitted in the completion.

prefill_tokens: usize

Number of tokens in the prompt (prefill phase).

time_to_first_token_ms: u64

Wall-clock milliseconds until the first token was emitted.

total_time_ms: u64

Total wall-clock milliseconds for the entire generation.

tokens_per_second: f32

Tokens-per-second throughput (cached result of StreamStats::throughput).

Implementations

impl StreamStats

pub fn new() -> Self

Create a blank StreamStats.

pub fn finish( &mut self, tokens: usize, prefill: usize, ttft_ms: u64, total_ms: u64, )

Record the final statistics after generation completes.

pub fn throughput(&self) -> f32

Compute tokens-per-second from recorded statistics.

Returns 0.0 if total_time_ms is zero (avoids division by zero).

pub fn to_usage_chunk(&self, request_id: &str, model: &str) -> String

Serialize these statistics as an SSE usage chunk.

The payload follows the OpenAI convention of appending a final usage chunk before [DONE]:

{"id":"...","object":"chat.completion.chunk","usage":{"prompt_tokens":…,…}}

Trait Implementations

impl Debug for StreamStats

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for StreamStats

fn default() -> StreamStats

Returns the “default value” for a type.

impl Serialize for StreamStats

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.