---
title: Requests
description: Every ChatRequest and Message field in rtb-chat, the wire shape each produces, and the defaults applied when you leave one unset.
tags: [reference, requests]
---
# Requests
`ChatRequest` is the argument to `chat`, `chat_stream` and `chat_structured`. It
derives `Default`, so the usual shape is to set the fields you care about:
```rust
use rtb_chat::{ChatRequest, Message, ThinkingMode};
let req = ChatRequest {
system: Some("You summarise changelogs.".into()),
messages: vec![Message::user("Summarise this changelog in one line.")],
max_tokens: Some(256),
cache_control: true,
thinking: Some(ThinkingMode::budget(2048)),
..ChatRequest::default()
};
```
## ChatRequest fields at a glance
| `system` | `Option<String>` | `None` | top-level `system` block | genai system prompt |
| `messages` | `Vec<Message>` | empty | `messages` array | genai messages |
| `temperature` | `Option<f32>` | `None` | `temperature` | dropped |
| `max_tokens` | `Option<u32>` | `None` → **1024** | `max_tokens` | dropped |
| `cache_control` | `bool` | `false` | `cache_control` markers | ignored |
| `thinking` | `Option<ThinkingMode>` | `None` | `thinking` block | ignored |
## max_tokens defaults to 1024 when unset
On the Anthropic-direct path the Messages API requires `max_tokens`, so
`rtb-chat` supplies `1024` when the field is `None`. That default is invisible
until a long reply gets cut off mid-sentence.
It bites hardest in `chat_structured`, where a truncated reply is not valid JSON
and comes back as `AiError::Deserialize` rather than anything that says "too
long". If a structured type has more than a handful of fields, set `max_tokens`
explicitly.
On the genai path `max_tokens` is not sent at all, and the provider's own default
applies.
## temperature and max_tokens are dropped on genai providers
Neither field reaches the wire on the genai backend. A request built with
`temperature: Some(0.2)` and `max_tokens: Some(64)` against
`Provider::OpenAiCompatible` produces this body:
```json
{"messages":[{"content":"hi","role":"user"}],"model":"llama-3.1-70b-instruct","stream":false}
```
No error, no warning — the model answers at the provider's defaults. This is a
defect rather than a design decision; see
[What rtb-chat does not do](../explanation/03-limitations.md#temperature-and-max_tokens-are-silently-dropped-on-genai-providers).
## Message, Role and ContentBlock
```rust
pub struct Message {
pub role: Role, // System | User | Assistant
pub content: Vec<ContentBlock>,
}
```
`Message::user`, `Message::system` and `Message::assistant` each build a message
with a single `ContentBlock::Text`.
`ContentBlock` has exactly one variant today, `Text(String)`, and is
`#[non_exhaustive]` so image and tool-use blocks can be added without a breaking
change. `ContentBlock::as_text()` returns `Option<&str>` for that reason; today it
is always `Some`.
Multiple blocks in one message are preserved on the Anthropic-direct path as
separate `{"type":"text"}` entries. On the genai path they are **joined with
newlines into a single string**, because genai's message type takes one body.
## A System role inside messages is not a system prompt
The two backends handle it differently, and neither does what the name suggests.
- **Anthropic-direct** — the Messages API has no `system` role in its `messages`
array, so a `Role::System` message is sent with `"role": "user"`. Its text
reaches the model as user input.
- **genai** — a `Role::System` message is applied as genai's system prompt, which
**replaces** anything set through `ChatRequest::system` or by an earlier
`Role::System` message. Last one wins.
Put the system prompt in `ChatRequest::system` and keep `messages` to `User` and
`Assistant` turns.
## cache_control marks two breakpoints
Setting `cache_control: true` adds `"cache_control": {"type": "ephemeral"}` to:
1. the `system` block, when `system` is `Some`; and
2. the **first text block of the first message** in `messages`.
Nothing else is marked. Later messages cache off the implicit prefix rather than
carrying their own breakpoint.
The resulting body:
```json
{
"model": "claude-opus-4-7",
"max_tokens": 1024,
"system": [{"type": "text", "text": "you are helpful",
"cache_control": {"type": "ephemeral"}}],
"messages": [{"role": "user", "content": [
{"type": "text", "text": "hi", "cache_control": {"type": "ephemeral"}}]}]
}
```
Cache hits and writes come back in `Usage::cache_read_input_tokens` and
`Usage::cache_creation_input_tokens`. Both are `0` on every genai provider,
whether or not there was a cache.
## ThinkingMode sets a token budget
```rust
pub enum ThinkingMode {
Budget { max_tokens: u32 },
}
```
`ThinkingMode::budget(2048)` is the constructor. The enum is `#[non_exhaustive]`
and has one variant today.
It serialises to `{"type": "enabled", "budget_tokens": <max_tokens>}` on the
Anthropic-direct path. Anthropic recommends at least 1024 tokens for the mode to
be worth enabling; `rtb-chat` does not enforce a floor, so a budget of `1` is sent
as-is and rejected by the provider.
Thinking output arrives as `ChatStreamEvent::ThinkingToken` on the streaming path.
On the non-streaming path there is nowhere for it to go: `parse_chat_response`
keeps only `"type": "text"` blocks, so thinking blocks in the response body are
discarded and `ChatResponse` carries the reply alone.
### Set max_tokens above the thinking budget
`rtb-chat` sends `max_tokens` and `budget_tokens` as independent numbers and does
not check that one fits inside the other. Leaving `max_tokens` unset means the
reply cap is 1024, so asking for a 2048-token thinking budget builds a request
whose budget exceeds its own cap — the provider rejects it, and the failure
arrives as `AiError::Provider` with the provider's own wording. Set `max_tokens`
comfortably above the budget whenever `thinking` is `Some`.
## chat_structured rewrites the system prompt
`chat_structured::<T>()` takes any `T: DeserializeOwned + JsonSchema`. Before
sending, it generates `T`'s JSON Schema and appends an instruction block to
`ChatRequest::system`:
```text
<your system prompt>
You MUST respond with a single JSON value matching this schema. No prose, no code fences:
{ ...the generated schema... }
```
Your own system prompt is kept as the prefix. If `system` was `None`, the
instruction becomes the whole system prompt.
The reply is then parsed as JSON, validated against the schema, and deserialised
into `T`. Because the instruction asks for bare JSON, a model that wraps its
answer in a ```` ```json ```` fence fails at the parse step with
`AiError::Deserialize` — the schema never gets a look in.
Nothing strips a fence or retries. If a model keeps fencing its output, say so
again in your own `system` prompt — it is prepended, so it is the first thing the
model reads.