Skip to main content

LlmStreamExecutionNextFn

Type Alias LlmStreamExecutionNextFn 

Source
pub type LlmStreamExecutionNextFn = Arc<dyn Fn(LlmRequest) -> Pin<Box<dyn Future<Output = Result<LlmJsonStream>> + Send>> + Send + Sync>;
Expand description

Continuation type invoked by streaming LLM execution intercepts.

This callable represents the remainder of the streaming LLM execution chain and resolves to a stream of JSON response chunks.

Aliased Typeยง

pub struct LlmStreamExecutionNextFn { /* private fields */ }