---
title: Responses and streaming
description: ChatResponse, Usage, Citation and every ChatStreamEvent in rtb-chat — what is populated, what is dropped, and when the stream ends.
tags: [reference, streaming]
---
# Responses and streaming
## ChatResponse
```rust
pub struct ChatResponse {
pub message: Message,
pub usage: Usage,
pub citations: Vec<Citation>,
}
```
`message` always has `role: Role::Assistant` and exactly one
`ContentBlock::Text`. Where the provider returned several text blocks they are
concatenated, with no separator. Blocks of any other type — images, tool use,
thinking — are dropped rather than surfaced.
A response body with no `content` array at all is `AiError::Provider`:
```text
provider error: missing `content` array on response
```
## Usage
```rust
pub struct Usage {
pub input_tokens: u32,
pub output_tokens: u32,
pub cache_creation_input_tokens: u32,
pub cache_read_input_tokens: u32,
}
```
Any field the provider omits reads `0` — there is no "not reported" state, so `0`
means either zero or unknown.
- **Anthropic-direct, non-streaming** — all four fields come from the response
`usage` object.
- **genai** — `input_tokens` and `output_tokens` come from genai's prompt and
completion counts. The two cache fields are hard-coded to `0`, regardless of
provider.
- **Streaming, either backend** — see
[Streaming usage is reported as zero](#streaming-usage-is-reported-as-zero).
Counts arrive from the provider as `u64` and are narrowed to `u32` with an `as`
cast, which truncates rather than saturating. No real token count comes close to
`u32::MAX`, so this is theory rather than practice.
## Citation
```rust
pub struct Citation {
pub cited_text: String,
pub source: String,
pub start_index: Option<u32>,
pub end_index: Option<u32>,
}
```
Populated only on the Anthropic-direct backend, and only when the model emits
citations. Every genai provider returns an empty vector.
`source` is filled from the first of `document_title`, `file_path`, `source` that
the provider supplies; if none is present it is an empty string rather than
`None`. `start_index` and `end_index` come from `start_char_index` and
`end_char_index` and stay `None` when absent.
## ChatStream and ChatStreamEvent
`chat_stream` returns a `ChatStream`, which implements
`futures_util::Stream<Item = ChatStreamEvent>`. It is `Send` — safe to move into
`tokio::spawn` — and not `Sync`.
```rust
pub enum ChatStreamEvent {
Token(String),
ThinkingToken(String),
Done(Usage),
Error(AiError),
}
```
`#[non_exhaustive]`, so consumers need a `_` arm.
| `Token` | SSE `content_block_delta` / `text_delta` | genai content chunk |
| `ThinkingToken` | SSE `content_block_delta` / `thinking_delta` | genai reasoning chunk |
| `Done(Usage)` | SSE `message_stop` | genai stream end |
| `Error` | SSE `error` event, or a transport failure mid-stream | any genai stream error |
Events `rtb-chat` does not model — `message_start`, `message_delta`, genai's
start and tool-call chunks — consume their bytes and produce nothing. Empty
`Token` events are filtered out on the genai path, so a genuinely empty text chunk
is dropped too.
## Done is not the end of the stream
`Done` is emitted when the terminating event arrives, but the stream keeps
yielding until the underlying byte stream closes. `while let Some(event) =
stream.next().await` will sit there until the connection ends.
Break out of the loop when you see `Done`:
```rust
use futures_util::StreamExt;
use rtb_chat::ChatStreamEvent;
async fn drain(mut stream: rtb_chat::ChatStream) -> String {
let mut answer = String::new();
while let Some(event) = stream.next().await {
match event {
ChatStreamEvent::Token(t) => answer.push_str(&t),
ChatStreamEvent::ThinkingToken(_) => {}
ChatStreamEvent::Done(_usage) => break,
ChatStreamEvent::Error(e) => { eprintln!("stream failed: {e}"); break }
_ => {}
}
}
answer
}
```
An `Error` event ends the exchange in practice — nothing useful follows it — but
`rtb-chat` does not close the stream for you.
## Streaming usage is reported as zero
`Done` on the Anthropic-direct path reads the token counts from the `message_stop`
SSE event. The Messages API does not put usage there: input counts arrive on
`message_start` and output counts on `message_delta`, both of which `rtb-chat`
parses as "no event" and discards.
The genai path lands in the same place by a different route — it reads genai's
`captured_usage`, which genai fills in only when `capture_usage` is enabled in its
chat options, and `rtb-chat` passes none.
The result is that `Done(usage)` on a real streaming call carries
`Usage::default()` — four zeros — even when the provider reported real numbers
earlier in the stream. Confirmed against a stream carrying
`message_start` with `input_tokens: 11` and `message_delta` with
`output_tokens: 42`:
```text
Done usage: Usage { input_tokens: 0, output_tokens: 0,
cache_creation_input_tokens: 0, cache_read_input_tokens: 0 }
```
This is a defect, and it is tracked as such — see
[What rtb-chat does not do](../explanation/03-limitations.md#streaming-token-counts-are-always-zero).
Until it is fixed, use the non-streaming `chat` call when you need to account for
tokens, or count them yourself.
## Errors during a stream
Errors split by when they happen:
- **Before the stream starts** — connection failures and non-2xx status codes are
returned from `chat_stream` itself, as a `Result::Err`. The same status mapping
as `chat` applies, so a 429 here is a real `AiError::RateLimited`.
- **During the stream** — surfaced as `ChatStreamEvent::Error` inside the stream.
These are always `AiError::Provider` (an SSE `error` event) or
`AiError::Transport` (the byte stream failed).