Expand description
Response accumulation and parsing utilities.
Handles both streaming (SSE) and non-streaming JSON response formats,
accumulating chunks into a unified ResponsePayload structure.
Streaming path uses a channel + spawn_blocking so that SSE JSON parsing
runs on a blocking thread while the async task continues reading from the
network — keeping the tokio executor thread free between chunk arrivals.
Structs§
- Response
Accumulator - Accumulates LLM response chunks from streaming or non-streaming sources.