pub struct MiddlewareChatModel { /* private fields */ }Expand description
A ChatModel wrapper that runs every call through a
MiddlewareChain’s model hooks.
Construct per node (or per invocation) — typically with the chain from
crate::node::NodeContext::middleware. The optional thread/node
labels flow into the ModelCall context.
chat_stream needs no override: the ChatModel default falls back to
chat, so streamed calls are intercepted identically.
Implementations§
Trait Implementations§
Source§impl ChatModel for MiddlewareChatModel
impl ChatModel for MiddlewareChatModel
Source§fn chat<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
messages: &'life1 [ChatMessage],
tools: &'life2 [Value],
) -> Pin<Box<dyn Future<Output = Result<ChatResponse>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
fn chat<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
messages: &'life1 [ChatMessage],
tools: &'life2 [Value],
) -> Pin<Box<dyn Future<Output = Result<ChatResponse>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Produce the next assistant message for the conversation.
Source§fn effect(&self) -> Effect
fn effect(&self) -> Effect
The declared effect classification of calling this model (Flight
Recorder, R0.5): recorded on model-call journal events and used by
retry/replay policy. Read more
Source§fn chat_stream<'life0, 'life1, 'life2, 'life3, 'async_trait>(
&'life0 self,
messages: &'life1 [ChatMessage],
tools: &'life2 [Value],
on_token: &'life3 mut (dyn FnMut(TokenChunk) + Send),
) -> Pin<Box<dyn Future<Output = Result<ChatResponse>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
fn chat_stream<'life0, 'life1, 'life2, 'life3, 'async_trait>(
&'life0 self,
messages: &'life1 [ChatMessage],
tools: &'life2 [Value],
on_token: &'life3 mut (dyn FnMut(TokenChunk) + Send),
) -> Pin<Box<dyn Future<Output = Result<ChatResponse>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
Produce the next assistant message, streaming token deltas through
on_token as they arrive. Read moreAuto Trait Implementations§
impl !RefUnwindSafe for MiddlewareChatModel
impl !UnwindSafe for MiddlewareChatModel
impl Freeze for MiddlewareChatModel
impl Send for MiddlewareChatModel
impl Sync for MiddlewareChatModel
impl Unpin for MiddlewareChatModel
impl UnsafeUnpin for MiddlewareChatModel
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more