pub struct RequestSample {
pub index: usize,
pub worker: usize,
pub start_s: f64,
pub end_s: f64,
pub token_times_s: Vec<f64>,
pub generated_tokens: u32,
pub prompt_tokens: u32,
pub outcome: Outcome,
pub in_flight_at_start: usize,
pub drained: bool,
}Expand description
One retained per-request sample (§4.4.5). Every time is an offset in seconds from a single band-wide origin, so the wall-clock span is computable across workers without reconstructing anything.
Fields§
§index: usizeMonotone index in issue order.
worker: usizeWhich closed-loop worker issued it.
start_s: f64Offset of the request start from the band origin.
end_s: f64Offset of the request’s completion (or abandonment) from the band origin.
token_times_s: Vec<f64>Arrival offsets of each streamed token. Empty when not streaming.
generated_tokens: u32Generated (completion) tokens, counted per the receipt’s tokenization block.
prompt_tokens: u32Prompt tokens, when the server reports them.
outcome: OutcomeHow the request ended.
in_flight_at_start: usizeConcurrent requests in flight at the instant this one was issued. The direct, per-request evidence that the client was actually concurrent.
drained: boolTrue when this request completed after the measurement window closed, i.e. during the §4.4.7 drain.
Implementations§
Source§impl RequestSample
impl RequestSample
Sourcepub fn ttft_ms(&self) -> Option<f64>
pub fn ttft_ms(&self) -> Option<f64>
§4.4.3 ttft_ms — request start to first token byte at the client.
None when no token was ever observed.
Sourcepub fn decode_tok_s(&self) -> Option<f64>
pub fn decode_tok_s(&self) -> Option<f64>
§4.4.3 decode_tok_s for this request:
(generated tokens - 1) / (last token time - first token time).
None when fewer than two tokens were observed, where the quantity is
undefined rather than zero.
Sourcepub fn itl_gaps_ms(&self) -> Vec<f64>
pub fn itl_gaps_ms(&self) -> Vec<f64>
§4.4.3 itl_ms — this request’s inter-token gaps, for pooling.
Sourcepub fn counts_toward_aggregate(&self) -> bool
pub fn counts_toward_aggregate(&self) -> bool
True when this sample contributes to agg_tok_s’s numerator (§4.4.3:
completed and non-truncated).
Trait Implementations§
Source§impl Clone for RequestSample
impl Clone for RequestSample
Source§impl Debug for RequestSample
impl Debug for RequestSample
Source§impl<'de> Deserialize<'de> for RequestSample
impl<'de> Deserialize<'de> for RequestSample
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Source§impl PartialEq for RequestSample
impl PartialEq for RequestSample
Source§impl Serialize for RequestSample
impl Serialize for RequestSample
impl StructuralPartialEq for RequestSample
Auto Trait Implementations§
impl Freeze for RequestSample
impl RefUnwindSafe for RequestSample
impl Send for RequestSample
impl Sync for RequestSample
impl Unpin for RequestSample
impl UnsafeUnpin for RequestSample
impl UnwindSafe for RequestSample
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> DeserializeOwned for Twhere
T: for<'de> Deserialize<'de>,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more