Expand description
Streaming state types: StreamState and ToolCallState.
StreamState was historically a single struct with ~30 fields covering
seven distinct concerns. It is now decomposed into single-responsibility
sub-structs (still pub for convenience at the call sites — the grouping
itself documents intent and lifecycle):
TextAccumulator— text/refusal/reasoning content + the thinking-tag buffer used by the streaming parser.IndexAllocator—output_index/content_indexallocator state and the spec-requiredsequence_numbercounter.UsageMetrics— token counts received from upstream.ToolCallTracker— in-flight (current) vs finalised (completed) function-call state.EmitState— boolean flags that drive the streaming emit state machine + the final response status / incomplete reason.
request_context no longer lives on StreamState; callers pass it as a
parameter to StreamState::build_response_object so that non-streaming
flows don’t have to construct a StreamState just to carry context.
Structs§
- Emit
State - Booleans driving the streaming emit state machine.
- Index
Allocator - Output-index / content-index / sequence-number allocators for the stream’s lifetime.
- Stream
State - Streaming converter state for tracking incremental changes.
- Text
Accumulator - Accumulated text content emitted across stream chunks.
- Tool
Call State - Tool
Call Tracker - In-flight + finalised function-call state.
- Usage
Metrics - Token usage observed from upstream chunks.