Skip to main content

ExperimentMonitor

Struct ExperimentMonitor 

Source
pub struct ExperimentMonitor { /* private fields */ }
Expand description

Per-(tenant, route) certified-selection state. Cheap to share behind an Arc.

Implementations§

Source§

impl ExperimentMonitor

Source

pub fn from_config(config: &Config) -> Self

A monitor for every route in config that has a challenger. Routes without one are ignored by every method, so a deployment with no experiments pays nothing.

Source

pub fn unavailable(config: &Config) -> Self

A monitor that refuses to serve any challenger. Used when the history could not be read.

Source

pub fn with_log(self, log: Arc<dyn ExperimentLogBackend>) -> Self

Attach a shared log backend. Every observe_outcome/observe_pair call will append to the backend, and sync_from_backend will replay newly-arrived records from it. Returns self for method-chaining.

Source

pub fn fingerprint(&self, route_ix: usize) -> Option<&str>

The fingerprint receipts for route_ix must carry to count, if it has a challenger.

Source

pub fn challenger_blocked(&self, tenant: &str, route_ix: usize) -> bool

Whether the serving path must NOT use the challenger for (tenant, route_ix): the experiment is halted, or its history was unreadable at startup.

Source

pub fn is_active(&self) -> bool

Whether any route runs an experiment.

Source

pub fn status(&self, tenant: &str, route_ix: usize) -> ExperimentStatus

Where the experiment for (tenant, route_ix) stands. ExperimentStatus::Undecided when nothing has been observed or the route has no challenger. The serving path stops using the challenger when this is ExperimentStatus::Halted.

Source

pub fn observe_cost( &self, tenant: &str, route_ix: usize, fingerprint: &str, arm: Arm, usd: f64, )

Record the cost of one served request of arm. Call with trace.final_.total_cost_usd.

fingerprint is the receipt’s; a mismatch with the current config counts the receipt in ignored_stale and records nothing.

Source

pub fn observe_health( &self, tenant: &str, route_ix: usize, fingerprint: &str, arm: Arm, errored: bool, truncated: u64, )

Tally one request of arm that errored and/or produced answers cut off at the output-token limit (truncated counts attempts). Stale fingerprints are ignored: observe_cost already counts the receipt once.

Source

pub fn observe_outcome( &self, tenant: &str, route_ix: usize, fingerprint: &str, arm: Arm, session_id: &str, verdict: Verdict, ) -> bool

Record a session’s task outcome. Returns whether it was counted.

Only the first clear outcome per session counts; later ones are ignored. The test pairs the k-th incumbent outcome with the k-th challenger outcome and treats sessions as independent draws, so several correlated outcomes from one long agent run would both overweight that session and break the independence the error bound rests on.

Pass is 1.0, Fail is 0.0, Abstain carries no information and is ignored.

Source

pub fn reports(&self) -> Vec<Snapshot>

A snapshot of every experiment, ordered by (tenant, route_ix).

Source

pub async fn sync_from_backend( &self, tenant: &str, route_ix: usize, fingerprint: &str, )

Read records appended to the shared log since the last sync and apply them locally.

Each call is cheap when there are no new records: one LRANGE that returns empty. Call this after every feedback event (within the feedback handler, not on the hot serve path) and from a background timer so all replicas stay ≤ 5 s stale.

No-op when no log backend is configured.

Source

pub async fn record_outcome( &self, tenant: &str, route_ix: usize, fingerprint: &str, arm: Arm, session_id: &str, verdict: Verdict, cost_usd: f64, ) -> bool

Record an outcome to the shared log (if configured), then sync from the log.

In single-replica mode (no log), falls back to the synchronous Self::observe_outcome. cost_usd is stored in the log so other replicas can tally costs without needing the full trace. Returns true when the outcome was counted.

Source

pub async fn record_pair( &self, tenant: &str, route_ix: usize, fingerprint: &str, x_inc: f64, x_ch: f64, incumbent_cost_usd: f64, challenger_cost_usd: f64, ) -> bool

Record a paired shadow result to the shared log (if configured), then sync.

In single-replica mode (no log), applies the pair directly and emits the status log. Returns true if x_inc / x_ch are valid (finite, in [0, 1]) and the pair was accepted by the underlying Experiment.

Source

pub fn start_sync_task(self: &Arc<Self>, interval: Duration)

Spawn a background task that calls sync_from_backend for every known experiment at most every interval (spec: ≤ 5 s). The task exits silently when the Arc becomes the last reference. No-op when no log backend is configured.

Trait Implementations§

Source§

impl Debug for ExperimentMonitor

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for ExperimentMonitor

Source§

fn default() -> Self

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<Unshared, Shared> IntoShared<Shared> for Unshared
where Shared: FromUnshared<Unshared>,

Source§

fn into_shared(self) -> Shared

Creates a shared type from an unshared type.
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more