Struct StatsResponse 

pub struct StatsResponse {
    pub uptime_seconds: u64,
    pub requests_total: u64,
    pub errors_total: u64,
    pub cache_hits: u64,
    pub cache_misses: u64,
    pub tokens_prompt_total: u64,
    pub tokens_generated_total: u64,
    pub last_request_age_seconds: Option<f64>,
    pub generating_now: usize,
    pub queue_depth: Option<usize>,
    pub queue_rejected_total: Option<u64>,
    pub recent: Vec<RecentRequest>,
}

Fields

uptime_seconds: u64

requests_total: u64

errors_total: u64

cache_hits: u64

cache_misses: u64

tokens_prompt_total: u64

tokens_generated_total: u64

last_request_age_seconds: Option<f64>

Seconds since the last request finished; null when none has.

generating_now: usize

Streamed generations decoding right now – the ones that could be stopped by POST /v1/cancel at this instant.

Not a queue depth: nothing is queued in front of a decode here, so this counts work in progress, not work waiting. Named for what it is so no one reads a backlog into it.

queue_depth: Option<usize>

Requests waiting for a decode slot, from the continuous-batching scheduler’s own queue.

null – not 0 – when continuous batching is off, because then there is no queue at all: every request goes straight onto its own blocking thread. A gauge reading 0 claims an empty queue was measured; null says there was nothing to measure, and a UI must be able to tell those apart.

queue_rejected_total: Option<u64>

Requests the queue turned away because it was full, since start. null under the same condition as Self::queue_depth.

recent: Vec<RecentRequest>

Newest last, capped server-side. See RecentRequest.

Trait Implementations

impl Clone for StatsResponse

fn clone(&self) -> StatsResponse

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for StatsResponse

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for StatsResponse

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for StatsResponse

fn eq(&self, other: &StatsResponse) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for StatsResponse

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for StatsResponse