Trait StreamingPromptHook 

pub trait StreamingPromptHook<M>:
    Clone
    + Send
    + Sync
where
    M: CompletionModel,
{
    // Provided methods
    fn on_completion_call(
        &self,
        prompt: &Message,
        history: &[Message],
        cancel_sig: CancelSignal,
    ) -> impl Future<Output = ()> + Send { ... }
    fn on_text_delta(
        &self,
        text_delta: &str,
        aggregated_text: &str,
        cancel_sig: CancelSignal,
    ) -> impl Future<Output = ()> + Send { ... }
    fn on_tool_call_delta(
        &self,
        tool_call_id: &str,
        tool_name: Option<&str>,
        tool_call_delta: &str,
        cancel_sig: CancelSignal,
    ) -> impl Future<Output = ()> + Send { ... }
    fn on_stream_completion_response_finish(
        &self,
        prompt: &Message,
        response: &<M as CompletionModel>::StreamingResponse,
        cancel_sig: CancelSignal,
    ) -> impl Future<Output = ()> + Send { ... }
    fn on_tool_call(
        &self,
        tool_name: &str,
        tool_call_id: Option<String>,
        args: &str,
        cancel_sig: CancelSignal,
    ) -> impl Future<Output = ()> + Send { ... }
    fn on_tool_result(
        &self,
        tool_name: &str,
        tool_call_id: Option<String>,
        args: &str,
        result: &str,
        cancel_sig: CancelSignal,
    ) -> impl Future<Output = ()> + Send { ... }
}

Trait for per-request hooks to observe tool call events.

Provided Methods

fn on_completion_call( &self, prompt: &Message, history: &[Message], cancel_sig: CancelSignal, ) -> impl Future<Output = ()> + Send

Called before the prompt is sent to the model

fn on_text_delta( &self, text_delta: &str, aggregated_text: &str, cancel_sig: CancelSignal, ) -> impl Future<Output = ()> + Send

Called when receiving a text delta

fn on_tool_call_delta( &self, tool_call_id: &str, tool_name: Option<&str>, tool_call_delta: &str, cancel_sig: CancelSignal, ) -> impl Future<Output = ()> + Send

Called when receiving a tool call delta. tool_name is Some on the first delta for a tool call, None on subsequent deltas.

fn on_stream_completion_response_finish( &self, prompt: &Message, response: &<M as CompletionModel>::StreamingResponse, cancel_sig: CancelSignal, ) -> impl Future<Output = ()> + Send

Called after the model provider has finished streaming a text response from their completion API to the client.

fn on_tool_call( &self, tool_name: &str, tool_call_id: Option<String>, args: &str, cancel_sig: CancelSignal, ) -> impl Future<Output = ()> + Send

Called before a tool is invoked.

fn on_tool_result( &self, tool_name: &str, tool_call_id: Option<String>, args: &str, result: &str, cancel_sig: CancelSignal, ) -> impl Future<Output = ()> + Send

Called after a tool is invoked (and a result has been returned).

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementations on Foreign Types

impl<M> StreamingPromptHook<M> for ()

where M: CompletionModel,

Implementors