Struct RequestBuilder 

pub struct RequestBuilder<'a, PromptState = PromptMissing, RequestModelState = ModelMissing, ClientModelState = ModelMissing> { /* private fields */ }

Builder for a single AI generation request.

Created by Client::request. Chain overrides, supply a prompt, then call one terminal method. Anything you do not override is inherited from the client.

Terminal methods

Method	Returns	Runs registered tools
generate	Response	yes, until a final answer
generate_once	Response	no, one provider call
generate_structured	StructuredOutput<T>	yes
generate_structured_once	StructuredOutput<T>	no
generate_with_history	Response	yes
stream	stream of provider events	not supported
generate_stream_events	stream of high-level events	not supported
stream_accumulated	Response	not supported

Methods ending in _once make exactly one provider call and never execute tools.

Typestate

The terminal methods only exist once the builder has both a prompt and a model, so an incomplete request cannot be sent. A model comes either from the client’s default or from model; the prompt comes from prompt. If generate appears to be missing, one of those two is absent.

Examples

use rai_sdk::{ClientBuilder, GenerationConfig, Model};

let client = ClientBuilder::new()
    .from_env()
    .model(Model::gpt4o_mini())
    .build()?;

let response = client
    .request()
    .model(Model::claude_sonnet_46())                       // override the model
    .config(GenerationConfig::new().with_temperature(0.2))   // override sampling
    .no_tools()                                             // ignore client tools
    .prompt("Summarize the borrow checker.")
    .generate()
    .await?;

Implementations

impl<'a, PromptState, RequestModelState, ClientModelState> RequestBuilder<'a, PromptState, RequestModelState, ClientModelState>

pub fn model( self, model: Model, ) -> RequestBuilder<'a, PromptState, ModelReady, ClientModelState>

Override the model, and therefore the provider, for this request.

Takes precedence over the client’s default model. Calling this makes the builder model-ready even if the client has no default.

pub fn config(self, config: GenerationConfig) -> Self

Override generation settings for this request.

Replaces the client’s default GenerationConfig wholesale rather than merging with it, so include every setting you want.

pub fn retry_config(self, config: RetryConfig) -> Self

Override the retry configuration for this request.

pub fn no_retry(self) -> Self

Disable retries for this request.

pub fn prompt<P>( self, prompt: P, ) -> RequestBuilder<'a, PromptReady, RequestModelState, ClientModelState>

where P: Into<Prompt>,

Set the prompt or conversation history for this request.

Accepts anything convertible into a Prompt: a &str, a String, a single Message, a Vec<Message>, or a full Prompt with multi-turn history and multimodal content.

Examples

use rai_sdk::{Message, Prompt};

// Each of these is accepted by `prompt()`.
let _: Prompt = "a plain string".into();
let _: Prompt = Message::user("a single message").into();
let _: Prompt = vec![
    Message::system("You are terse."),
    Message::user("Explain lifetimes."),
]
.into();

// Or build one up explicitly.
let _ = Prompt::single(Message::system("You are terse."))
    .with_message(Message::user("Explain lifetimes."));

pub fn tool(self, tool: Tool) -> Self

Replace inherited tools with a single request-specific tool.

pub fn tools<T>(self, tools: T) -> Self

where T: IntoIterator<Item = Tool>,

Replace inherited tools with a custom set for this request.

pub fn additional_tool(self, tool: Tool) -> Self

Add one more tool while still keeping client-level tools.

pub fn additional_tools<T>(self, tools: T) -> Self

where T: IntoIterator<Item = Tool>,

Add several request-only tools while still keeping client-level tools.

pub fn no_tools(self) -> Self

Disable all tools for this request, including client defaults.

Also the way to stream from a client that has tools registered, since the streaming methods reject any request carrying tools.

impl<'a, ClientModelState> RequestBuilder<'a, PromptReady, ModelReady, ClientModelState>

pub async fn generate(self) -> Result<Response>

Generate a response, automatically executing any tool calls the model requests.

This is the method you usually want. If tools are registered, it runs the loop — send, execute requested tools, append results, send again — until the model answers without asking for more tools. With no tools registered, it is a single call.

Transient failures are retried according to the effective RetryConfig.

Errors

• Error::ProviderNotConfigured if the provider has no API key, or Error::ProviderNotEnabled if its Cargo feature is off.
• Error::ToolLoopLimitExceeded if the model keeps requesting tools past GenerationConfig::with_max_tool_rounds (default 8).
• Error::ToolNotFound if the model requests a tool that is not registered.
• Error::RateLimit, Error::Timeout, or Error::Http if the request still fails after retries.
• Error::Auth, Error::InvalidRequest, Error::ContentFiltered, or Error::Request for provider-side rejections.

Note that a tool handler returning an error does not fail this call: the error is passed back to the model as tool content so it can react.

Examples

use rai_sdk::{ClientBuilder, Model};

let client = ClientBuilder::new()
    .from_env()
    .model(Model::gpt4o_mini())
    .build()?;

let response = client
    .request()
    .prompt("Name one Rust testing crate.")
    .generate()
    .await?;

println!("{}", response.text());
if let Some(usage) = &response.usage {
    println!("tokens: {:?}", usage.total_tokens);
}

pub async fn generate_once(self) -> Result<Response>

Make exactly one provider call, without executing tools.

Tool definitions are still advertised to the model, so the response may contain tool calls — they are returned to you on the response messages instead of being executed. Use this when you want to inspect, gate, or approve tool calls, or drive the loop yourself.

Errors

Same as generate, except it cannot return Error::ToolLoopLimitExceeded or Error::ToolNotFound, since no tool is executed.

Examples

use rai_sdk::{ClientBuilder, Model};

let response = client
    .request()
    .prompt("What is the weather in Paris?")
    .generate_once()
    .await?;

for message in &response.messages {
    for call in &message.tool_calls {
        println!("requested {} with {}", call.name, call.arguments);
    }
}

pub async fn generate_structured<T>(self) -> Result<StructuredOutput<T>>

where T: DeserializeOwned + JsonSchema,

Generate a response that must match the Rust type T.

A JSON Schema is generated from T and sent to the provider, the response is validated against that schema, and only then deserialized. Tools still run as in generate.

T must be non-recursive: recursive types force $ref/$defs, which strict providers reject. See GenerationConfig::with_json_schema_for.

Errors

Everything generate can return, plus Error::StructuredOutput if the response is empty, is not valid JSON, fails schema validation, or does not deserialize into T.

Examples

use rai_sdk::{ClientBuilder, JsonSchema, Model};
use serde::Deserialize;

#[derive(Debug, Deserialize, JsonSchema)]
struct Summary {
    title: String,
    bullet_points: Vec<String>,
}

let structured = client
    .request()
    .prompt("Summarize the Rust ownership model.")
    .generate_structured::<Summary>()
    .await?;

println!("{}", structured.output.title);
for point in &structured.output.bullet_points {
    println!("- {point}");
}

pub async fn generate_structured_once<T>(self) -> Result<StructuredOutput<T>>

where T: DeserializeOwned + JsonSchema,

Make exactly one provider call and parse the result as T.

Unlike generate_once, configured tools are not even advertised to the model: they are ignored entirely (and a log line records that). Use this for a pure transformation on a client that happens to have tools registered.

Errors

Same as generate_structured, minus the tool-loop errors.

pub async fn generate_with_history( self, history: &[ConversationTurn], ) -> Result<Response>

Generate a response with prior conversation turns prepended.

A convenience over assembling the history into the Prompt yourself: each ConversationTurn contributes its user message, assistant message, and any tool results, followed by this request’s prompt. Tools run as in generate.

Errors

Same as generate.

pub async fn generate_stream_events( self, ) -> Result<impl Stream<Item = Result<StreamEvent>> + Send>

Stream the response as high-level StreamEvents.

Higher level than stream: text deltas are passed through, tool-call argument fragments are buffered and emitted as whole calls, and a final TurnComplete event carries the assembled ConversationTurn — convenient for feeding conversation history back into a later request.

Registered tools are not executed; this only reports what the model asked for.

To forward these events to a remote client instead of consuming them in process, see stream_wire_events; WireStreamEvent also implements From<StreamEvent> if you would rather convert these.

Cancellation

Dropping the returned stream aborts the upstream provider request. See the “Cancellation” section of stream.

Errors

Same as stream, including Error::InvalidRequest when the request’s effective tool set is non-empty. Once the stream is open, individual items may also be errors.

pub async fn stream_wire_events( self, ) -> Result<Pin<Box<dyn Stream<Item = WireStreamEvent> + Send>>>

Stream the response as serializable WireStreamEvents, ready to forward to a remote client.

This is the SDK half of the proxy pattern: your server holds the provider credentials, calls this, and re-emits each event as an SSE data: payload; the client parses them back into WireStreamEvents and rebuilds the response with StreamAccumulator. See the wire module for the format and its compatibility guarantees, and examples/sse_proxy.rs for the whole loop.

Stream shape

Unlike the other streaming methods, items are not Results. Once the stream is open every outcome is an event, so a mid-stream provider failure reaches the client as WireStreamEvent::Error instead of as a silently truncated response. The sequence is:

1. exactly one MessageStart;
2. any number of text and tool-call events;
3. one Usage, when the provider reported token counts;
4. exactly one terminal event — MessageStop on success, Error on failure.

Tool-call arguments are reported twice over: incrementally as ToolCallStart plus ToolCallDelta so a UI can render progress, then once assembled as ToolCallEnd. A client that only wants finished calls can ignore the first two.

Registered tools are not executed, exactly as with generate_stream_events.

Cancellation

Dropping the returned stream aborts the upstream provider request. See the “Cancellation” section of stream — it matters more here than anywhere else, because for a proxy the consumer being dropped is the end client hanging up.

Errors

The returned Result covers only failures that happen before the stream opens: the same causes as stream, including Error::InvalidRequest when the request’s effective tool set is non-empty. A server that wants its client to see those too can forward them with WireStreamEvent::error.

Examples

use futures::StreamExt;
use rai_sdk::{ClientBuilder, Model};

let mut events = client
    .request()
    .prompt("Summarize the news.")
    .stream_wire_events()
    .await?;

while let Some(event) = events.next().await {
    // `data: {"type":"text_delta","text":"..."}`
    println!("data: {}\n", serde_json::to_string(&event)?);
}

pub async fn stream( self, ) -> Result<Pin<Box<dyn Stream<Item = Result<ProviderStreamEvent>> + Send>>>

Stream raw provider events as they arrive.

Use this to render output incrementally. Each item is a Result, since a stream can fail partway through — do not discard the error case, or a mid-stream failure will look like a clean end of output.

Cancellation

Dropping the stream aborts the upstream provider request. Every streaming method in this crate is driven entirely by the consumer: the provider’s HTTP response body is polled from inside the returned stream, never from a detached background task. Dropping the stream therefore drops the response body and closes the underlying connection, and the provider stops generating. Nothing keeps running in the background and no tokens are burned on output nobody will read.

Two consequences worth planning for:

• A generation cancelled this way produces no terminal event — no Done, no usage. Providers bill for what they generated before the abort, so a server that meters usage cannot rely on the final usage event alone.
• Cancellation propagates through wrappers. Dropping the future or stream returned by generate_stream_events, stream_wire_events, or stream_accumulated — including when the whole task is cancelled by tokio::time::timeout or by an axum client disconnect — aborts the provider request just the same.

Errors

Returns Error::InvalidRequest if the request would carry any tool, because streaming cannot run a tool loop. This considers the request’s effective tool set, so no_tools lets you stream from a client that has tools registered, and tool on the request is rejected even when the client itself has none.

Otherwise the same causes as Client::generate_stream: Error::ProviderNotConfigured, Error::ProviderNotEnabled, or a transport or provider failure. Once the stream is open, individual items may also be errors.

Examples

use futures::StreamExt;
use rai_sdk::{ClientBuilder, Model, provider::ProviderStreamEvent};

let mut stream = client
    .request()
    .prompt("Count from one to five.")
    .stream()
    .await?;

while let Some(event) = stream.next().await {
    match event? {
        ProviderStreamEvent::Text(text) => print!("{text}"),
        ProviderStreamEvent::Done { .. } => println!(),
        _ => {}
    }
}

pub async fn stream_accumulated(self) -> Result<Response>

Stream internally and return one complete Response.

Uses the streaming transport (lower time-to-first-byte, and less likely to sit near a timeout on long generations) but consumes every chunk for you, so the result is shaped exactly like generate. Reach for this when you want streaming’s latency behavior without handling events.

Only text and the terminating event are accumulated, so tool calls are not represented in the returned response.

Cancellation

Dropping the returned future aborts the upstream provider request. See the “Cancellation” section of stream.

Errors

Same as stream, including Error::InvalidRequest when the request’s effective tool set is non-empty, plus any error encountered while consuming the stream.

Examples

use rai_sdk::{ClientBuilder, Model};

let response = client
    .request()
    .prompt("Write a short launch announcement.")
    .stream_accumulated()
    .await?;

println!("{}", response.text());