Struct ModelStats 

pub struct ModelStats {
    pub inference_count: u64,
    pub total_inference_time_ms: f64,
    pub last_inference_time: Option<SystemTime>,
    pub average_inference_time_ms: f64,
    pub error_count: u64,
    pub success_count: u64,
    pub min_inference_time_ms: Option<f64>,
    pub max_inference_time_ms: Option<f64>,
    pub batch_count: u64,
    pub batched_items: u64,
}

A model’s request counters, as /metrics reports them.

Counts are per request; timings are per forward pass, so requests batched together each record the time of the pass they shared.

Fields

inference_count: u64

Requests answered, successfully or not

total_inference_time_ms: f64

Sum of the forward-pass times recorded for successful requests

last_inference_time: Option<SystemTime>

When the last successful request finished; RFC 3339 in JSON

average_inference_time_ms: f64

total_inference_time_ms / success_count

error_count: u64

Requests that failed, whether their inputs were invalid or the model failed

success_count: u64

Requests that got their outputs

min_inference_time_ms: Option<f64>

Fastest forward pass a successful request was part of

max_inference_time_ms: Option<f64>

Slowest forward pass a successful request was part of

batch_count: u64

Forward passes run. Below success_count when requests were batched together

batched_items: u64

Items processed by those forward passes

Trait Implementations

impl Clone for ModelStats

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelStats

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ModelStats

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for ModelStats

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.