Skip to main content

Module messages_stream

Module messages_stream 

Source
Expand description

Streaming Messages-native gateway tool loop.

Consumes vLLM’s per-round Anthropic SSE and presents the client one logical message across all gateway rounds:

  • message_start emitted once (first round only);
  • surfaced content_block_* forwarded with client-visible indices rebased contiguously across rounds;
  • gateway-owned tool_use blocks suppressed (and their input_json_delta buffered to reconstruct the call for dispatch);
  • intermediate message_delta/message_stop (the per-round terminals) suppressed; the final round’s terminal is forwarded once.

Structurally the Anthropic-native analogue of the Responses GatewayStreamAccumulator (#119/#132); kept deliberately parallel for a future consolidation. Reuses only the neutral tool layer via crate::types::messages::tool_seam.

Functions§

run_messages_stream
Drive the streaming Messages-native loop, yielding Anthropic SSE lines for the client. Owns the multi-round → single-message accumulation.