pub type LlmStreamExecutionNextFn = Arc<dyn Fn(LlmRequest) -> Pin<Box<dyn Future<Output = Result<LlmJsonStream>> + Send>> + Send + Sync>;Expand description
Continuation type invoked by streaming LLM execution intercepts.
This callable represents the remainder of the streaming LLM execution chain and resolves to a stream of JSON response chunks.
§Parameters
- First argument: LLM request to pass to the remaining streaming execution chain.
§Returns
A future resolving to a JSON chunk stream.
§Errors
The future resolves to an error when the remaining streaming execution chain fails.
§Lifetime
This continuation can be called repeatedly or concurrently while its
execution-intercept callback is still running. Each invocation receives an
isolated snapshot of the scopes visible when next is called. Calls that
remain unfinished or begin after the interceptor settles are rejected.
Returning an interceptor stream extends that active lifetime until the
stream closes, which permits lazy stream adapters to call next while they
are being consumed. A stream successfully returned by next keeps its
ordinary stream lifetime.
Aliased Type§
pub struct LlmStreamExecutionNextFn { /* private fields */ }