Struct ChatRequestBuilder 

pub struct ChatRequestBuilder { /* private fields */ }

Available on crate features client or blocking only.

Builder for LLM proxy chat requests (async + streaming).

Implementations

impl ChatRequestBuilder

pub fn message(self, role: MessageRole, content: impl Into<String>) -> Self

Add a message with the given role and content.

pub fn system(self, content: impl Into<String>) -> Self

Add a system message.

pub fn user(self, content: impl Into<String>) -> Self

Add a user message.

pub fn assistant(self, content: impl Into<String>) -> Self

Add an assistant message.

pub fn messages(self, messages: Vec<ProxyMessage>) -> Self

Set the full message list, replacing any existing messages.

pub fn max_tokens(self, max_tokens: i64) -> Self

Set the maximum number of tokens to generate.

pub fn temperature(self, temperature: f64) -> Self

Set the sampling temperature.

pub fn metadata(self, metadata: HashMap<String, String>) -> Self

Set request metadata.

pub fn metadata_entry( self, key: impl Into<String>, value: impl Into<String>, ) -> Self

Add a single metadata entry. Empty keys or values are ignored.

pub fn response_format(self, response_format: ResponseFormat) -> Self

Set the response format (e.g., JSON schema for structured outputs).

pub fn stop(self, stop: Vec<String>) -> Self

Set stop sequences.

pub fn tools(self, tools: Vec<Tool>) -> Self

Set tools available for the model to call.

pub fn tool_choice(self, tool_choice: ToolChoice) -> Self

Set the tool choice strategy.

pub fn request_id(self, request_id: impl Into<String>) -> Self

Set a request ID for tracing.

pub fn header(self, key: impl Into<String>, value: impl Into<String>) -> Self

Add a custom header to the request.

pub fn timeout(self, timeout: Duration) -> Self

Set the request timeout.

pub fn retry(self, retry: RetryConfig) -> Self

Set retry configuration.

impl ChatRequestBuilder

pub fn new(model: impl Into<Model>) -> Self

Create a new chat request builder for the given model.

pub fn build_request(&self) -> Result<ProxyRequest, Error>

pub async fn send(self, client: &LLMClient) -> Result<ProxyResponse, Error>

Available on crate feature client only.

Execute the chat request (non-streaming, async).

pub async fn stream(self, client: &LLMClient) -> Result<StreamHandle, Error>

Available on crate features client and streaming only.

Execute the chat request and stream responses (async).

pub async fn stream_deltas( self, client: &LLMClient, ) -> Result<Pin<Box<dyn Stream<Item = Result<String, Error>> + Send>>, Error>

Available on crate features client and streaming only.

Execute the chat request and stream text deltas (async).

pub fn structured<T>(self) -> StructuredChatBuilder<T>

where T: JsonSchema + DeserializeOwned,

Available on crate feature client only.

Create a structured output builder with automatic schema generation.

This method transitions the builder to a [StructuredChatBuilder] that automatically generates a JSON schema from the type T and handles validation retries.

Example

use schemars::JsonSchema;
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize, JsonSchema)]
struct Person {
    name: String,
    age: u32,
}

let result = client.chat()
    .model("claude-sonnet-4-20250514")
    .user("Extract: John Doe, 30 years old")
    .structured::<Person>()
    .max_retries(2)
    .send()
    .await?;

pub async fn stream_json<T>( self, client: &LLMClient, ) -> Result<StructuredJSONStream<T>, Error>

where T: DeserializeOwned,

Available on crate features client and streaming only.

Execute the chat request and stream structured JSON payloads (async).

The request must include a structured response_format (type=json_schema), and uses NDJSON framing per the /llm/proxy structured streaming contract.

Trait Implementations

impl Clone for ChatRequestBuilder

fn clone(&self) -> ChatRequestBuilder

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ChatRequestBuilder

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ChatRequestBuilder

fn default() -> ChatRequestBuilder

Returns the “default value” for a type.