Trait InspectableBackendSession 

pub trait InspectableBackendSession<B: BackendProvider>: BackendSession<B> {
    // Required methods
    fn inspect_prefill(
        &mut self,
        backend: &B,
        input: Self::PrefillInput,
        request: &ObservationRequest,
    ) -> Result<InspectedOutput<Self::Output>, B::Error>;
    fn inspect_decode(
        &mut self,
        backend: &B,
        input: Self::DecodeInput,
        request: &ObservationRequest,
    ) -> Result<InspectedOutput<Self::Output>, B::Error>;
}

Optional named-activation inspection for a selected backend session.

This is a general diagnostics and observability capability. Implementations execute the requested operation to completion and materialize only selected observation points. It is intentionally separate from ordinary asynchronous submission so production inference pays no instrumentation cost.

Required Methods

fn inspect_prefill( &mut self, backend: &B, input: Self::PrefillInput, request: &ObservationRequest, ) -> Result<InspectedOutput<Self::Output>, B::Error>

Executes and inspects one prompt prefill operation.

fn inspect_decode( &mut self, backend: &B, input: Self::DecodeInput, request: &ObservationRequest, ) -> Result<InspectedOutput<Self::Output>, B::Error>

Executes and inspects one cached decode operation.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors