---
title: Responses and streaming
description: 'ChatResponse, Usage, Citation and every ChatStreamEvent in rtb-chat: what is populated, what is dropped, and when the stream ends.'
tags: [reference, streaming]
---
# Responses and streaming
## ChatResponse
```rust
pub struct ChatResponse {
pub message: Message,
pub usage: Usage,
pub citations: Vec<Citation>,
}
```
`message` always has `role: Role::Assistant` and exactly one
`ContentBlock::Text`. Where the provider returned several text blocks they are
concatenated, with no separator. Blocks of any other type (images, tool use,
thinking) are dropped rather than surfaced.
A response body with no `content` array at all is `AiError::Provider`:
```text
provider error: missing `content` array on response
```
## Usage
```rust
pub struct Usage {
pub input_tokens: u32,
pub output_tokens: u32,
pub cache_creation_input_tokens: u32,
pub cache_read_input_tokens: u32,
}
```
Any field the provider omits reads `0`. There is no "not reported" state, so `0`
means either zero or unknown.
- **Anthropic-direct, non-streaming**: all four fields come from the response
`usage` object.
- **genai**: `input_tokens` and `output_tokens` come from genai's prompt and
completion counts. The two cache fields are hard-coded to `0`, regardless of
provider.
- **Streaming, either backend**: see
[Streaming usage arrives on Done](#streaming-usage-arrives-on-done).
Counts arrive from the provider as `u64` and are narrowed to `u32` with an `as`
cast, which truncates rather than saturating. No real token count comes close to
`u32::MAX`, so this is theory rather than practice.
## Citation
```rust
pub struct Citation {
pub cited_text: String,
pub source: String,
pub start_index: Option<u32>,
pub end_index: Option<u32>,
}
```
Populated only on the Anthropic-direct backend, and only when the model emits
citations. Every genai provider returns an empty vector.
`source` is filled from the first of `document_title`, `file_path`, `source` that
the provider supplies; if none is present it is an empty string rather than
`None`. `start_index` and `end_index` come from `start_char_index` and
`end_char_index` and stay `None` when absent.
## ChatStream and ChatStreamEvent
`chat_stream` returns a `ChatStream`, which implements
`futures_util::Stream<Item = ChatStreamEvent>`. It is `Send`, so it can move into
`tokio::spawn`, and it is not `Sync`.
```rust
pub enum ChatStreamEvent {
Token(String),
ThinkingToken(String),
Done(Usage),
Error(AiError),
}
```
`#[non_exhaustive]`, so consumers need a `_` arm.
| `Token` | SSE `content_block_delta` / `text_delta` | genai content chunk |
| `ThinkingToken` | SSE `content_block_delta` / `thinking_delta` | genai reasoning chunk |
| `Done(Usage)` | SSE `message_stop` | genai stream end |
| `Error` | SSE `error` event, or a transport failure mid-stream | any genai stream error |
Events `rtb-chat` does not model (`message_start`, `message_delta`, genai's
start and tool-call chunks) consume their bytes and produce nothing. Empty
`Token` events are filtered out on the genai path, so a genuinely empty text chunk
is dropped too.
## Done is not the end of the stream
`Done` is emitted when the terminating event arrives, but the stream keeps
yielding until the underlying byte stream closes. `while let Some(event) =
stream.next().await` will sit there until the connection ends.
Break out of the loop when you see `Done`:
```rust
use futures_util::StreamExt;
use rtb_chat::ChatStreamEvent;
async fn drain(mut stream: rtb_chat::ChatStream) -> String {
let mut answer = String::new();
while let Some(event) = stream.next().await {
match event {
ChatStreamEvent::Token(t) => answer.push_str(&t),
ChatStreamEvent::ThinkingToken(_) => {}
ChatStreamEvent::Done(_usage) => break,
ChatStreamEvent::Error(e) => { eprintln!("stream failed: {e}"); break }
_ => {}
}
}
answer
}
```
An `Error` event ends the exchange in practice, since nothing useful follows it, but
`rtb-chat` does not close the stream for you.
## Streaming usage arrives on Done
`Done(usage)` carries the counts the provider reported during the stream.
On the **Anthropic-direct** path the Messages API spreads them over two events.
`message_start` carries the input and cache counts, and `message_delta` carries
the running output count. `rtb-chat` keeps the latest of each and puts them on
`Done` when `message_stop` arrives.
On the **genai** path `rtb-chat` switches on genai's `capture_usage` for every
stream, and `Done` carries what genai captured. For the OpenAI adapter that means
the request asks for `stream_options: {"include_usage": true}`, so an
OpenAI-compatible server must accept that field. As on the non-streaming genai
path, the two cache fields are always `0`.
A provider that reports nothing during the stream still gives you four zeros.
## Errors during a stream
Errors split by when they happen:
- **Before the stream starts**: connection failures and non-2xx status codes are
returned from `chat_stream` itself, as a `Result::Err`. On the Anthropic-direct
backend the same status mapping as `chat` applies, so a 429 here is a real
`AiError::RateLimited`; on genai it is `AiError::Provider`, as it is for `chat`.
- **During the stream**: surfaced as `ChatStreamEvent::Error` inside the stream.
These are always `AiError::Provider` (an SSE `error` event) or
`AiError::Transport` (the byte stream failed).