pub struct EndpointPhases {
pub decode_ns: u64,
pub user_ns: u64,
pub encode_ns: u64,
pub queue_ns: u64,
pub adapter_ns: u64,
pub held_episodes: Option<u32>,
pub held_state_bytes: Option<u64>,
pub in_flight: u32,
pub lane_skew_ns: Option<u64>,
}Expand description
The endpoint-local split of JoinResponse.endpoint_total_ns (ns), plus the
pre-handler wait and slot depth a pipelining model endpoint reports.
A zero duration is “not measured”: a peer built before these fields existed
sends none of them, and a reader sees this default. The gauges are Option
because for them a measured zero is a real sample (an even vector has zero
skew; an engine that just evicted holds zero episodes) that must stay
distinguishable from a peer that never measures. in_flight is model-only;
lane_skew_ns is env-only.
Fields§
§decode_ns: u64Decoding the request off the wire.
user_ns: u64The env or model implementation’s own work, user code included.
encode_ns: u64Encoding the response onto the wire.
queue_ns: u64Wait before the handler ran. Model: concurrency permit, route gate, handler lock. Env: the env lock, which serializes ops on one env.
adapter_ns: u64Adapter work inside user_ns (observation assembly + action apply) —
a sub-span, not additive: the model’s own forward is user_ns - adapter_ns. Model-only, and zero for a spec-less route.
held_episodes: Option<u32>Episodes whose frame-stack windows the endpoint’s adapter engine held
when the response was stamped, across all routes. Model-only; None
from a handler that keeps no such accounting.
held_state_bytes: Option<u64>Bytes those held frame-stack windows occupy. Model-only; None from a
handler that keeps no such accounting.
in_flight: u32Requests holding a concurrency slot when this request was dispatched to its handler (>= 1): slot occupancy, not parallelism.
lane_skew_ns: Option<u64>Straggler skew across a vector env’s lanes for this op — see
lane_skew_ns. Env-only; None from an env that does not time its
lanes, Some(0) from one whose lanes were even.
Implementations§
Source§impl EndpointPhases
impl EndpointPhases
Sourcepub fn reported(ns: u64) -> Option<u64>
pub fn reported(ns: u64) -> Option<u64>
Wire form of a measurement: an unmeasured (zero) phase is left off the
message rather than sent as a 0 a reader cannot tell from silence.
Sourcepub fn from_durations(
decode: Duration,
user: Duration,
encode: Duration,
) -> EndpointPhases
pub fn from_durations( decode: Duration, user: Duration, encode: Duration, ) -> EndpointPhases
Build a split from measured phase durations, saturating at u64::MAX ns.
Sourcepub fn nest(
decode_ns: u64,
call_ns: u64,
encode_ns: u64,
inner: EndpointPhases,
) -> EndpointPhases
pub fn nest( decode_ns: u64, call_ns: u64, encode_ns: u64, inner: EndpointPhases, ) -> EndpointPhases
Fold an inner layer’s own split into this layer’s decode/call/encode
measurement. The inner layer’s decode and encode accumulate into the
outer ones, and user_ns keeps the rest of the call — the inner user
work plus whatever it cost to reach it (GIL wait, adapter overhead) —
so the phases still sum to this layer’s whole measurement. An inner
layer that measures nothing, or whose wire share does not even fit
inside the measured call, leaves the whole call as user time.
Sourcepub fn from_env_response(response: &JoinResponse) -> EndpointPhases
pub fn from_env_response(response: &JoinResponse) -> EndpointPhases
Read the split an env peer stamped; all-zero for a peer that sends none.
Sourcepub fn from_model_response(response: &JoinResponse) -> EndpointPhases
pub fn from_model_response(response: &JoinResponse) -> EndpointPhases
Read the split a model peer stamped; all-zero for a peer that sends none.
Trait Implementations§
Source§impl Clone for EndpointPhases
impl Clone for EndpointPhases
Source§fn clone(&self) -> EndpointPhases
fn clone(&self) -> EndpointPhases
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreimpl Copy for EndpointPhases
Source§impl Debug for EndpointPhases
impl Debug for EndpointPhases
Source§impl Default for EndpointPhases
impl Default for EndpointPhases
Source§fn default() -> EndpointPhases
fn default() -> EndpointPhases
impl Eq for EndpointPhases
Source§impl PartialEq for EndpointPhases
impl PartialEq for EndpointPhases
impl StructuralPartialEq for EndpointPhases
Auto Trait Implementations§
impl Freeze for EndpointPhases
impl RefUnwindSafe for EndpointPhases
impl Send for EndpointPhases
impl Sync for EndpointPhases
impl Unpin for EndpointPhases
impl UnsafeUnpin for EndpointPhases
impl UnwindSafe for EndpointPhases
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key and return true if they are equal.Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
Source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
T in a tonic::Request