Expand description
Unified interceptor system for LLM calls and tool executions.
This module provides a composable middleware-like system that works across
different domains. The core abstraction is the Interceptor trait, which
wraps operations and can inspect, modify, or short-circuit them.
§Architecture
InterceptorStack::new()
.with(Logging::default()) // outermost: sees request first
.with(Retry::default()) // retries wrap everything inside
.with(Timeout::new(30s)) // timeout on inner operation
.execute(&input, operation)§Domains
The system supports multiple domains via marker types:
ToolExec<Ctx>- Tool executions (integrated withToolRegistry)LlmCall- LLM provider requests (reserved for future use, not yet integrated)
§Generic Interceptors
Interceptors like Retry, Timeout, and Logging are generic over
any domain that implements the required behavior traits (Retryable,
Timeoutable, Loggable).
§Example
ⓘ
use llm_stack::ToolRegistry;
use llm_stack::intercept::{InterceptorStack, Retry, Timeout, ToolExec};
use std::time::Duration;
let registry: ToolRegistry<()> = ToolRegistry::new()
.with_interceptors(
InterceptorStack::<ToolExec<()>>::new()
.with(Retry::default())
.with(Timeout::new(Duration::from_secs(30)))
);Re-exports§
pub use behavior::Loggable;pub use behavior::Outcome;pub use behavior::Retryable;pub use behavior::Timeoutable;pub use domain::LlmCall;pub use domain::ToolExec;pub use domain::ToolRequest;pub use domain::ToolResponse;pub use interceptors::LogLevel;pub use interceptors::Logging;pub use interceptors::NoOp;pub use interceptors::Retry;pub use interceptors::Timeout;pub use tool_interceptors::Approval;pub use tool_interceptors::ApprovalDecision;
Modules§
- behavior
- Behavior traits that interceptors can require.
- domain
- Domain-specific marker types and their Interceptable implementations.
- interceptors
- Built-in interceptors for common cross-cutting concerns.
- tool_
interceptors - Tool-specific interceptors.
Structs§
- FnOperation
- Wrap a closure as an
Operation. - Interceptor
Stack - A composable stack of interceptors.
- Next
- Handle to invoke the next interceptor in the chain (or the final operation).
Traits§
- Interceptable
- An operation that can be intercepted.
- Interceptor
- Wraps an interceptable operation.
- Operation
- The final operation to execute after all interceptors.