Skip to main content

Module streamed

Module streamed 

Source
Expand description

Streamed-turn assembly for AgentRun.

A streamed model turn arrives as incremental StreamedAssistantContent items. StreamedTurnAssembler is the sans-IO accumulator that turns that item stream into the same canonical complete turn the non-streaming path feeds the machine — while telling the driver what to forward to its consumer and surfacing invalid tool calls the moment they appear, so a driver can stop paying for a doomed provider stream early.

The protocol, paired with the streamed entry points on AgentRun:

  1. On AgentRunStep::CallModel, open a provider stream and create one assembler per turn with the tool names advertised for that turn.
  2. Feed every stream item to StreamedTurnAssembler::ingest and act on the returned StreamedTurnEvents: forward items to the consumer, and on StreamedTurnEvent::InvalidToolCall consult AgentRun::resolve_streamed_invalid_tool_callStreamedResolution::Repaired continues the same stream via StreamedTurnAssembler::resolve_pending_invalid; StreamedResolution::TurnAbandoned means drain the provider stream for usage and re-enter AgentRun::next_step.
  3. When the provider stream ends, call StreamedTurnAssembler::finish and feed the result to AgentRun::streamed_turn; the run then proceeds exactly like a non-streamed one (CallTools / Done).

crate::streaming::StreamingPrompt::stream_prompt drives this protocol internally; hand-driven runs can use it to stream any AgentRun.

Structs§

PartialStreamedTurn
Snapshot of a streamed turn at the moment an invalid tool call appeared. Used by the machine to build diagnostics and rollback messages from exactly what the model has produced so far.
StreamedInvalidToolCall
One invalid tool call surfaced mid-stream, awaiting a resolution from AgentRun::resolve_streamed_invalid_tool_call.
StreamedTurn
The assembled streamed turn, fed to AgentRun::streamed_turn.
StreamedTurnAssembler
Sans-IO accumulator that assembles one streamed model turn. See the module docs for the driving protocol.

Enums§

StreamedResolution
What the machine decided about a mid-stream invalid tool call.
StreamedTurnEvent
What a driver must do with one ingested stream item.