pub fn run_tool_loop_stream<'a>(
model: &'a dyn TextModel,
prompt: Prompt,
executor: &'a dyn ToolExecutor,
budget: ToolBudget,
) -> BoxStream<'a, Result<ToolLoopEvent, ToolLoopError>>Expand description
Streams prompt to completion through model, running any tool calls
with executor within budget (issue #859) — the streaming
counterpart of run_tool_loop. The gates before the first call, the
budgets, and the item that ends the stream are the buffered loop’s own:
a ToolLoopEvent::Done carrying the same ToolLoopOutcome the
buffered loop would have returned, or an Err carrying the same
ToolLoopError. What differs is the plumbing. Each step calls
TextModel::stream instead of TextModel::complete, and the work
comes out as ToolLoopEvents as it happens: every TextDelta the
moment it arrives (through a bounded channel, so a slow consumer
backpressures the model rather than queueing it), one
ToolLoopEvent::ToolResult per executed call, and the step’s
StepUsage once its usage is known. The step’s Completion is
rebuilt from its deltas with CompletionBuilder, and the decision
both loops then make — usage bookkeeping, answer-or-run, the turns
appended to the transcript — is one shared function, so they cannot
drift.
Structured output is the one behavioural difference, and it is the
port’s own: the streamed outcome’s Completion::json is None even
for a prompt carrying Prompt::json_schema, because the parsed value
is a buffered complete feature — CompletionBuilder documents why
a relayed stream cannot promise it. text holds the same answer
either loop saw, and nothing else about the outcome differs; a
streaming caller wanting the value parses and validates the
reassembled text against the prompt’s schema itself, exactly as the
stream half of the port tells its callers.
§Cancellation
Dropping the stream drops the loop and with it the in-flight model
stream — which aborts the upstream exchange, the contract
TextModel::stream carries — so no further tool runs and no further
model call is made.
§Errors
The same cases run_tool_loop documents, reported as the stream’s
last item instead of a return value: ToolLoopError::Model for a
refused prompt or a model that failed — including one that fails
mid-stream, where the step’s earlier deltas have already been forwarded
and the error is the one a failed complete would have produced —
ToolLoopError::InvalidToolSchema before the first call, and the two
budget errors once spent. After an error item the stream ends.