Skip to main content

run_tool_loop_stream

Function run_tool_loop_stream 

Source
pub fn run_tool_loop_stream<'a>(
    model: &'a dyn TextModel,
    prompt: Prompt,
    executor: &'a dyn ToolExecutor,
    budget: ToolBudget,
) -> BoxStream<'a, Result<ToolLoopEvent, ToolLoopError>>
Expand description

Streams prompt to completion through model, running any tool calls with executor within budget (issue #859) — the streaming counterpart of run_tool_loop. The gates before the first call, the budgets, and the item that ends the stream are the buffered loop’s own: a ToolLoopEvent::Done carrying the same ToolLoopOutcome the buffered loop would have returned, or an Err carrying the same ToolLoopError. What differs is the plumbing. Each step calls TextModel::stream instead of TextModel::complete, and the work comes out as ToolLoopEvents as it happens: every TextDelta the moment it arrives (through a bounded channel, so a slow consumer backpressures the model rather than queueing it), one ToolLoopEvent::ToolResult per executed call, and the step’s StepUsage once its usage is known. The step’s Completion is rebuilt from its deltas with CompletionBuilder, and the decision both loops then make — usage bookkeeping, answer-or-run, the turns appended to the transcript — is one shared function, so they cannot drift.

Structured output is the one behavioural difference, and it is the port’s own: the streamed outcome’s Completion::json is None even for a prompt carrying Prompt::json_schema, because the parsed value is a buffered complete feature — CompletionBuilder documents why a relayed stream cannot promise it. text holds the same answer either loop saw, and nothing else about the outcome differs; a streaming caller wanting the value parses and validates the reassembled text against the prompt’s schema itself, exactly as the stream half of the port tells its callers.

§Cancellation

Dropping the stream drops the loop and with it the in-flight model stream — which aborts the upstream exchange, the contract TextModel::stream carries — so no further tool runs and no further model call is made.

§Errors

The same cases run_tool_loop documents, reported as the stream’s last item instead of a return value: ToolLoopError::Model for a refused prompt or a model that failed — including one that fails mid-stream, where the step’s earlier deltas have already been forwarded and the error is the one a failed complete would have produced — ToolLoopError::InvalidToolSchema before the first call, and the two budget errors once spent. After an error item the stream ends.