Skip to main content

Module state

Module state 

Source
Expand description

Streaming state types: StreamState and ToolCallState.

StreamState was historically a single struct with ~30 fields covering seven distinct concerns. It is now decomposed into single-responsibility sub-structs (still pub for convenience at the call sites — the grouping itself documents intent and lifecycle):

  • TextAccumulator — text/refusal/reasoning content + the thinking-tag buffer used by the streaming parser.
  • IndexAllocatoroutput_index / content_index allocator state and the spec-required sequence_number counter.
  • UsageMetrics — token counts received from upstream.
  • ToolCallTracker — in-flight (current) vs finalised (completed) function-call state.
  • EmitState — boolean flags that drive the streaming emit state machine + the final response status / incomplete reason.

request_context no longer lives on StreamState; callers pass it as a parameter to StreamState::build_response_object so that non-streaming flows don’t have to construct a StreamState just to carry context.

Structs§

EmitState
Booleans driving the streaming emit state machine.
IndexAllocator
Output-index / content-index / sequence-number allocators for the stream’s lifetime.
StreamState
Streaming converter state for tracking incremental changes.
TextAccumulator
Accumulated text content emitted across stream chunks.
ToolCallState
ToolCallTracker
In-flight + finalised function-call state.
UsageMetrics
Token usage observed from upstream chunks.