Skip to main content

ProviderMiddleware

pe_core::provider_middleware

Trait ProviderMiddleware 

Source 
pub trait ProviderMiddleware:
    Send
    + Sync
    + 'static {
    // Required method
    fn wrap_complete<'life0, 'life1, 'life2, 'life3, 'async_trait>(
        &'life0 self,
        messages: &'life1 [Message],
        tools: &'life2 [ToolSchema],
        next: &'life3 dyn LlmProvider,
    ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, PeError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             'life3: 'async_trait;
}
Expand description

Intercepts LLM completion calls for cross-cutting concerns.

Middlewares are composed into a MiddlewareStack and execute outside-in: the first middleware added wraps all subsequent ones.

§Implementors

Required Methods§

Source

fn wrap_complete<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, messages: &'life1 [Message], tools: &'life2 [ToolSchema], next: &'life3 dyn LlmProvider, ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, PeError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Wrap a completion call. Call next.complete(messages, tools).await to forward to the next layer (or the base provider).

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§