llmkit-tower 0.1.0

Tower middleware (retry, rate limit, cost tracking, tracing) for llmkit-rs
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

//! The [`LlmLayer`] trait: a Tower-style decorator over [`LlmProvider`].

use std::sync::Arc;

use llmkit_core::LlmProvider;

/// Wraps an inner [`LlmProvider`] in a new one, adding cross-cutting behaviour.
///
/// Analogous to `tower::Layer`, but specialised to the LLM provider trait so
/// that wrapped providers remain `LlmProvider`s — no `Service<Request>` generic
/// gymnastics or sprawling `where` clauses. Each layer's output is itself a
/// provider, so layers compose freely and the result is a single
/// `Arc<dyn LlmProvider>`.
pub trait LlmLayer {
    /// The provider produced by wrapping `inner`.
    type Provider: LlmProvider;

    /// Wrap `inner`, returning the decorated provider.
    fn layer(self, inner: Arc<dyn LlmProvider>) -> Self::Provider;
}