Struct ChatRequest 

#[non_exhaustive]
pub struct ChatRequest {
    pub model: ModelId,
    pub system: Option<String>,
    pub messages: Vec<Message>,
    pub tools: Vec<ToolSchema>,
    pub generation: Generation,
    pub thinking: Thinking,
    pub response_schema: Option<Value>,
    pub cache_breakpoints: Vec<usize>,
}

One request to one model.

Build it with ChatRequest::new and the with_ methods. The struct is marked non exhaustive, so fields can be added without breaking your code, and constructing it with a literal will not compile from outside this crate.

use llmr::{ChatRequest, Message};

let request = ChatRequest::new("claude-sonnet-5", vec![Message::user("Hello")])
    .with_system("You answer in one sentence.")
    .with_max_tokens(256);

assert_eq!(request.model.as_str(), "claude-sonnet-5");

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

model: ModelId

Which model.

system: Option<String>

Instructions that are not part of the conversation.

messages: Vec<Message>

The conversation so far, oldest first.

tools: Vec<ToolSchema>

Tools the model may call.

generation: Generation

Sampling settings.

thinking: Thinking

Whether to reason before answering.

response_schema: Option<Value>

A JSON Schema the reply must match.

Providers that cannot do this say so through crate::ModelCapabilities::structured_output. Check before you set it.

cache_breakpoints: Vec<usize>

Where to place cache breakpoints, counted in messages from the start.

A provider without prompt caching ignores this. It costs money to get wrong in either direction, so it is explicit rather than inferred.

Implementations

impl ChatRequest

pub fn new(model: impl Into<ModelId>, messages: Vec<Message>) -> Self

A request with nothing set beyond the model and the conversation.

pub fn with_system(self, system: impl Into<String>) -> Self

Sets the system instructions.

pub fn with_tools(self, tools: Vec<ToolSchema>) -> Self

Offers the model a set of tools.

pub fn with_max_tokens(self, max: u32) -> Self

Caps the reply length.

pub fn with_temperature(self, temperature: f32) -> Self

Sets the sampling temperature.

pub fn with_thinking(self, thinking: Thinking) -> Self

Asks the model to reason before answering.

pub fn with_response_schema(self, schema: Value) -> Self

Requires the reply to match a schema.

pub fn with_cache_breakpoint(self, after_message: usize) -> Self

Places a cache breakpoint after the message at this index.

pub fn needs(&self) -> Needs

What this request needs a model to support.

Compare it against crate::Provider::capabilities to find out what will be dropped before you send anything.

Trait Implementations

impl Clone for ChatRequest

fn clone(&self) -> ChatRequest

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ChatRequest

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ChatRequest

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for ChatRequest

fn eq(&self, other: &ChatRequest) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for ChatRequest

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for ChatRequest