Struct StreamingPromptRequest 

pub struct StreamingPromptRequest<M, P>
where
    M: CompletionModel,
    P: StreamingPromptHook<M> + 'static,
{ /* private fields */ }

A builder for creating prompt requests with customizable options. Uses generics to track which options have been set during the build process.

If you expect to continuously call tools, you will want to ensure you use the .multi_turn() argument to add more turns as by default, it is 0 (meaning only 1 tool round-trip). Otherwise, attempting to await (which will send the prompt request) can potentially return crate::completion::request::PromptError::MaxDepthError if the agent decides to call tools back to back.

Implementations

impl<M, P> StreamingPromptRequest<M, P>

where M: CompletionModel + 'static, <M as CompletionModel>::StreamingResponse: WasmCompatSend + GetTokenUsage, P: StreamingPromptHook<M>,

pub fn new( agent: Arc<Agent<M>>, prompt: impl Into<Message>, ) -> StreamingPromptRequest<M, P>

Create a new PromptRequest with the given prompt and model

pub fn multi_turn(self, depth: usize) -> StreamingPromptRequest<M, P>

Set the maximum depth for multi-turn conversations (ie, the maximum number of turns an LLM can have calling tools before writing a text response). If the maximum turn number is exceeded, it will return a crate::completion::request::PromptError::MaxDepthError.

pub fn with_history(self, history: Vec<Message>) -> StreamingPromptRequest<M, P>

Add chat history to the prompt request

pub fn with_hook<P2>(self, hook: P2) -> StreamingPromptRequest<M, P2>

where P2: StreamingPromptHook<M>,

Attach a per-request hook for tool call events

Trait Implementations

impl<M, P> IntoFuture for StreamingPromptRequest<M, P>

where M: CompletionModel + 'static, <M as CompletionModel>::StreamingResponse: WasmCompatSend, P: StreamingPromptHook<M> + 'static,

type Output = Pin<Box<dyn Stream<Item = Result<MultiTurnStreamItem<<M as CompletionModel>::StreamingResponse>, StreamingError>> + Send>>

The output that the future will produce on completion.

type IntoFuture = Pin<Box<dyn Future<Output = <StreamingPromptRequest<M, P> as IntoFuture>::Output> + Send>>

Which kind of future are we turning this into?

fn into_future(self) -> <StreamingPromptRequest<M, P> as IntoFuture>::IntoFuture

Creates a future from a value.